tsvstat

Generates a table of summary statistics in TSV format given a table of numerical observations in TSV format.

cat tests/test1.tsv | tsvstat.exe

To build

$ cargo build
$ cargo tests
$ cargo install

Summary statistics

These statistics are calculated on streaming data:

• Count
• Sum
• Range, min, max
• Mean
• Standard deviation
• Variance
• Skew
• Kurtosis and excess Kurtosis
• Coefficient of Variation

Numeric type

One of the following is displayed to indicating the type of the data in each column: Binary, +Integer, -Integer, Integer, -Real, +Real, Real or Not Numeric.

Future additions

• Distribution tests such as jarque barra normality test
• Covariance table
• Correlation table
• Multiple regression - choose one table as the independent variable or y and the others as dependent variables or x's

Robust statistics - Median, Quartiles and Percentiles

One cannot always calculates the median, quartiles or percentiles for two reasons, first it would require sorting the data first and secondly it would require holding the entire dataset in memory rather than just streaming inputs.

• Sorting - the code could easilly test if the data is already sorted and then calculate these statistics.
• Memory - perhaps another program could be used for this.