The unit DataTable implements an often needed data structure when working with measurement data. It combines a rectangular numeric matrix with additional information, such as row and column names, cell states, or attributes.

The numeric matrix is inherited from TMatrix, thus all mathematical methods of TMatrix can be used for the data table as well. In addition to the row and column names, each row and each column of the data table provides an 8-bit user-defined attribute which can be used, for example to introduce additional classification of the row entries. Besides an arbitrary comment which is assigned to the entire data table, each numeric cell is described by an 8-bit state flag, which can be used by the programmer to indicate any user-defined state of the cell.

More details can be found in the online help pages. The unit DataTable is part of the SDL Component Suite.

What's new:

• Release 10.7   [Aug-24, 2020]
   
  • the method ScanForCatValues analyses a categorical variable
  • the method Assign implemented
  • the new method RemoveMarkedRows removes all marked rows
  • the new methods CopyDataColumnFromArray, CopyDataColumnToArray, CopyDataRowFromArray and CopyDataRowToArray support the copying of columns and rows to and from open arrays
  • TDataTable.SumOfSquaredMarkedCells and TDatatable.SumOfSquaredNumCells calculate the sum of squared data cells
  • TDataTable.ExportAsASC and TDataTable.ImportASC now support the export and import of custom data
  • the function QuartilesOfMarkedCells calculates the quartiles of marked cells
  • the new methods CopyDataFromArray and CopyDataToArray allow to copy the numeric data to open arrays
  • new method IfAnyCellHasCellState allows to test cells states of a range of cells
  • the new method CreateContingencyTable calculates the contingency table
  • the method CountNominalIDs returns the number of defined nominal IDs
  • bug fix: the methods MinRowAttrib, MaxRowAttrib, MinColAttrib and MaxColAttrib returned an invalid row/column index under some circumstances
  
  
• Release 10.6   [Mar-09, 2018]
   
  • no changes
  
  
• Release 10.5   [Oct-10, 2016]
   
  • the procedures SetRowAttributes and SetColAttributes offer now an overloaded version which allows to mask out bits
  
  
• Release 10.4   [Jun-02, 2015]
   
  • DT_MaxNominalIDs has been increased to 50
  
  
• Release 10.3   [Oct-06, 2014]
   
  • the properties ColAttrib and RowAttrib have been changed from byte to integer in order to allow for more class numbers
  • MaxRowAttrib and MinRowAttrib return the maximum and minimum attributes of a range of rows
  • MaxColAttrib and MinColAttrib return the maximum and minimum attributes of a range of columns
  • IsConstantColumn detects whether a columns contains all constant values
  • the functions MaxColNameLength and MaxRowNameLength retrieve the maximum length of the column and the row names
  • the methods SetColAttributes and SetRowAttributes allow to set all column or row attributes of the data table
  • declaration of TDTxmlTags
  • the method ExportAsASC now comes in a third overloaded version supporting the export of data into an open text file
  • the methods SaveAsXMLFile and LoadFromXMLFile store and load the data table in XML format
  • the methods ReadFromOpenXMLFile, ReadFromXMLStream, and ReadFromXMLString .... read the data table from an XML file, from a stream, or from a string, respectively
  • the methods WriteToOpenXMLFile and WriteToXMLStream store the datatable in XML format on a file, or a stream, respectively
  
  
• Release 10.2   [May-30, 2013]
   
  • the method Serialize has been extended to include the column and row indices in the serialized matrix
  • the event OnPercentDone has been implemented to provide feedback during read and write operations of CSV files
  • the new function CopyCellsToVector copies numeric values of a subrange to a vector
  • the new function CopyFrom allows to copy part of an external data table
  
  
• Release 10.1   [Oct-29, 2012]
   
  • the new method CopyCellStatesFrom copies the cell states of an external table
  • the new method ListOfCategorialVars returns the list of categorial variables
  • the functions FindLowestMLevel and FindHighestMLevel return the lowest and the highest level of measurement in a series of columns
  • the new method Serialize allows to serialize the data table
  • the functions ColAttribLevels and RowAttribLevels count the number of attribute levels for the column and row attributes, respectively
  • the function PercentileOfMarkedCells calculates a particular percentile of marked cells
  • the method CheckDichotomousColumn checks for dichotomous variables
  • the method InvertCellStates inverts particular states of all cells
  • the event OnChangeNames is triggered when any of the column/row names are changed
  • the event OnChangeAttrib is triggered when any of the column/row attributes are changed
  • the event OnChangeCellState when the state of a cell is changed
  • new methods ChangedNames, ChangedCellStates, and ChangedAttrib trigger the OnChangeNames, OnChangeCellStates, and OnChangeAttrib events, respectively
  • the function FillMarkedCells allows to fill marked cells
  • The following methods now allow to select an entire row/column by setting the appropriate range to [0..0]: CountEmptyCells, CountImputedCells, CountNumCells, CountValidCells, FindHighestMLevel, GeometricMeanOfNumCells, HarmonicMeanOfNumCells, MeanVarOfMarkedCells, MeanVarOfNumCells, MinMaxOfMarkedCells, MinMaxOfNumCells, MinMaxOfValidCells, PercentileOfNumCells, QuartilesOfNumCells, SortColAttributes, SortColNames, SortCols, SortRowAttributes, SortRowNames, SortRows, SumOfMarkedCells, SumOfNumCells
  • the new method CalcCovar calculates the scatter/covariance/correlation matrix
  • an additional overloaded version of ExportAsCSV allows to include the comment in CSV files
  • the methods SortColNames and SortRowNames now provide a second overloaded version which allow to control lower/upper case sorting
  • the method ExchangeMarkings switches all markings of type A and B
  • bug fix: ExportAsCSV now correctly exports the column headers in the case the row headers are not included
  • bug fix: the following methods did not return the correct value for the variable parameter "NumData": GeometricMeanOfNumCells, HarmonicMeanOfNumCells, PercentileOfNumCells, QuartilesOfNumCells
  
  
• Release 10.0   [Oct-4, 2011]
   
  • a cell state csImputed introduced (replacing csUser4)
  • the method ExportAsCSV allows now to export row attributes, as well
  • the new method CountEmptyCells counts the number of empty cells
  • the new method CountImputedCells counts the number of imputed cells
  
  
• Release 9.7   [May-31, 2010]
   
  • the new methods CRCofCellStates and CRCofNamesAndAttributes allow to calculate a CRC code over the cell states and the names and attributes of the table
  • the methods ExchangeColumns and ExchangeRows allow to exchange two columns/rows
  • the following new methods can be applied to cells having a particular cell state: SumOfMarkedCells (sums up cells), MinMaxOfMarkedCells (searches the minimum and maximum), and MeanVarOfMarkedCells (calculates the mean and the standard deviation)
  • ClearNominalIDs clears all nominal IDs of a column
  • the methods MirrorColumns and MirrorRows allow to mirror a data table
  • new method CountValidCells counts all valid non-empty cells
  • new method MinMaxOfValidCells finds minimum and maximum of valid non-empty cells
  • the methods GeometricMeanOfNumCells, HarmonicMeanOfNumCells, MeanVarOfNumCells, QuartilesOfNumCells, CountNumCells, SumOfNumCells, MinMaxOfNumCells, and PercentileOfNumCells now account for categorial columns
  • ExportAsASC and ExportAsCSV are now functions returning an error message if the file cannot be created or overwritten
  • ImportASC and ReadFromASC now check if the file exists and return an error code if not
  • greatly improved speed of ExportAsASC for large data matrices
  • the method Changed has been renamed to ChangedData
  • new method ChangedSetup allows to trigger the OnChangeSetup event programmatically
  • new event OnChangeSetup is triggered when the layout of the table is changed
  • bug fix: ImportFromCSV did not reset scaling information of variables
  • bug fix: ImportFromCSV now marks empty cells as empty (instead of setting them to zero values)
  
  
• Release 9.51   [Dec-01, 2008]
   
  • no changes
  
  
• Release 9.5   [Oct-31, 2008]
   
  • data table now supports reading from CSV files and writing to CSV files - see ImportFromCSV and ExportAsCSV
  • additional version of ReadFromASC to read from clipboard
  • data table now supports columnwise measurement scales (interval, ratio, nominal, and categorial), see TMScaleType, MScaleType, NominalID, OrdinalOfNominalID, AddNominalID, and ElemNominal
  • new methods InsertRow, InsertColumn, RemoveRow, and RemoveColumn allow to insert, delete, and append entire rows and columns
  • new methods SortRowNames and SortRowAttributes sort the data according to the row names and the row attributes
  • new methods SortColNames and SortColAttributes sort the data according to the column names and the column attributes
  • method UnmarkAllCells resets all cell markings to zero values
  • new property IsEmpty allows to control the "empty" state of a cell
  • new property IsNumber tests a cell for a valid numeric value
  • export and import of ASC files now allow for empty column/row identifiers and empty or NAN data cells by storing a special empty cell identifier
  • bug fix: StandardizeNumCellColumns and StandardizeNumCellRows did not include the last row/column of the data matrix
  • bug fix: assigning a Comment does no longer create a range check error
  • QuartilesOfNumCells and PercentileOfNumCells calculate now more appropriate values for very small sample sizes
  • bug fix: row attributes are now unchanged if ReadFromASC is used with ReplaceIDs set to FALSE
  
  
• Release 9.0   [May-28, 2007]
   
  • new constants csAll, csNone, and csUser1..csUser4 defined to reflect predefined cell states
  • new functions CountMarkedCellsInColumn and CountMarkedCellsInRow count marked cells
  • new overloaded method ImportASC allows to read from clipboard
  • bug fix: ImportASC now does not crash if data is imported from stream and number of columns is reduced
  
  
• Release 8.5   [Feb-18, 2006]
   
  • ExportAsASC (DataTable): default width of exported data is now 12 characters
  • optimized resizing of the data table
  • several new methods to calculate statistical parameters of the numeric cells only: MeanVarOfNumCells, QuartilesOfNumCells, MinMaxOfNumCells, PercentileOfNumCells, GeometricMeanOfNumCells, HarmonicMeanOfNumCells, CountNumCells, SumOfNumCells, StandardizeNumCellColumns, StandardizeNumCellRows
  
  
• Release 8.2   [Nov-03, 2004]
   
  • no changes
  
  
• Release 8.1   [Aug-29, 2004]
   
  • available for Delphi™ 8 for .NET
  
  
• Release 8.0   [Apr-01, 2004]
   
  • DataTable is now available to the publis and part of MathPack