Q mode
info:update t:"V",rule:130,maybe:1b from info where t="I",mw in 5 6,7<count each dchar,{all x like"*[0-9][0-5][0-9][0-5][0-9]"}peach sdv,.csv.cancast["V"]peach sdv; / 235959 12345
1
/ utilities to quickly load a csv file - for more exhaustive analysis of the csv contents see csvguess.q
2
/ 2009.09.20 - updated to match latest csvguess.q
3
4
/ .csv.colhdrs[file] - return a list of colhdrs from file
5
/ info:.csv.info[file] - return a table of information about the file
6
/ columns are:
7
/ c - column name; ci - column index; t - load type; mw - max width;
8
/ dchar - distinct characters in values; rule - rule that caught the type
9
/ maybe - needs checking, _could_ be say a date, but perhaps just a float?
10
/ .csv.info0[file;onlycols] - like .csv.info except that it only analyses <onlycols>
11
/ example:
12
/ info:.csv.info0[file;(.csv.colhdrs file)like"*price"]
13
/ info:.csv.infolike[file;"*price"]
14
/ show delete from info where t=" "
15
/ .csv.data[file;info] - use the info from .csv.info to read the data
16
/ .csv.data10[file;info] - like .csv.data but only returns the first 10 rows
17
/ bulkload[file;info] - bulk loads file into table DATA (which must be already defined :: DATA:() )
18
/ .csv.read[file]/read10[file] - for when you don't care about checking/tweaking the <info> before reading
19
20
\d .csv
21
DELIM:","
22
ZAPHDRS:0b / lowercase and remove _ from colhdrs (junk characters are always removed)
23
WIDTHHDR:25000 / number of characters read to get the header
24
READLINES:222 / number of lines read and used to guess the types
25
SYMMAXWIDTH:11 / character columns narrower than this are stored as symbols
26
SYMMAXGR:10 / max symbol granularity% before we give up and keep as a * string
27
FORCECHARWIDTH:30 / every field (of any type) with values this wide or more is forced to character "*"
28
DISCARDEMPTY:0b / completely ignore empty columns if true else set them to "C"
29
CHUNKSIZE:50000000 / used in fs2 (modified .Q.fs)
30
31
k)nameltrim:{$[~@x;.z.s'x;~(*x)in aA:.Q.a,.Q.A;(+/&\~x in aA)_x;x]}
32
k)fs2:{[f;s]((-7!s)>){[f;s;x]i:1+last@&0xa=r:1:(s;x;CHUNKSIZE);f@`\:i#r;x+i}[f;s]/0j}
MIME type defined: text/x-q
.