Copyright © 2016 Ashok P. Nadkarni. All rights reserved.

1. Overview

As is true for scripting languages in general, Tcl incurs significant overhead in terms of memory as well as performance when operating on large amounts of data. The set of packages described here provide a more performant and convenient means of working with large data sets. The three packages are

  • tarray — implements the core data types and operations

  • xtal — provides succint, convenient syntax for working with data sets

  • tarray_ui — contains Tk widgets that are useful for display and user interaction

The packages come with comprehensive documentation, an extensive test suite and are covered by a liberal open source license.

Note The packages require Tcl version 8.6 or later.

2. The tarray package

The tarray (typed array) extension implements two new Tcl collection data types — columns and tables. Typed arrays store elements of a specific data type in native format and in comparison to Tcl collection types like lists and dicts, offer performance benefits of

  • efficient memory utilization

  • fast execution

  • parallelization on multi-core systems

In addition, the extension provides some convenience features that are not built into lists and dictionaries

  • vector operations

  • flexible indexing

  • more powerful search and sort facilities

The column and table constructs are similar to dataframes in languages like R.

3. The xtal package

The related extension xtal implements Xtal, a language geared towards vector operations on lists, columns and tables. Xtal can be embedded within Tcl in a similar fashion to the expr command or used to write entire procedures and scripts. Xtal adds a succint syntactic layer on top of the tarray extension. For example, the following Xtal script to retrieve employees in Boston within a specific salary range would replace several lines of Tcl.

Emps[Emps.Salary > 30000 && Emps.Salary < 50000 && Emps.Location == "Boston"]

Xtal can be used with standard Tcl structures like lists and dictionaries as well.

% set L [list 1.01 10.1 .101 101]
→ 1.01 10.1 .101 101
% xtal::xtal { L[L > 1 && L < 100] }
→ 1.01 10.1

See the Xtal Quick Tour for a short overview of the language.

You can also use Xtal in interactive mode, either at the tclsh prompt, the wish console or tkcon. See The Xtal shell for details.

4. The tarray_ui package

The tarray_ui package implements widgets that useful for displaying and interacting with typed arrays. Currently this only includes widgets for displaying table data with sorting/filtering capabilities and for importing CSV files.

5. Documentation guide

6. License

The packages are primarily covered by the liberal open source BSD license. See the file license.terms in the source directories for details. Some third-party components used in the software may have their own license.