This is a mountain tour of q, the programming language built into kdb+.It takes the form of a one-page ‘ridge walk’ along the tops, with optional side descents to see more detail.
Use it as a very fast start with kdb+, or for a quick overview of what it is like to work in q.
Overview
Before you start
Download, install and launch q.Experiment with the expressions as you read.
Followlinks to deeper treatments of topics.
The q session¶
The q session is a REPL.It evaluates a q expression and prints the result.You can use it as a calculator.
$ qKDB+ 3.7t 2020.03.05 Copyright (C) 1993-2020 Kx Systemsm64/ 4()core 8192MB sjt ...q)2+2 3 44 5 6q)acos -13.141593End your session with the Terminate system command.
q)\\$The q session
Databases¶
Tables¶
Tables are first-class objects in q.
Load the Suppliers and Parts database.
q)\l sp.q+`p`city!(`p$`p1`p2`p3`p4`p5`p6`p1`p2;`london`london`london`london`london`lon..(`s#+(,`color)!,`s#`blue`green`red)!+(,`qty)!,900 1000 1200+`s`p`qty!(`s$`s1`s1`s1`s2`s3`s4;`p$`p1`p4`p6`p2`p2`p4;300 200 100 400 200 300)q)\a`p`s`spThe \l system command loaded and ran the script sp.q. The script defined three tables, ran three queries against them and displayed the results.
The \a system command listed the names of tables.
q)sp / suppliers and partss p qty---------s1 p1 300s1 p2 200s1 p3 400s1 p4 200s4 p5 100s1 p6 100s2 p1 300s2 p2 400s3 p2 200s4 p2 200s4 p4 300s1 p5 400Tables
Save a table to a file of the same name.
q)save `:path/to/sp / kdb+ format`:path/to/spq)save `:path/to/sp.xls / Excel spreadsheet`:path/to/sp.xlsOther save formats include CSV, plain text and XML.
Large tables can be splayed (each column written as its own file) and partitioned into time periods.
CSVs¶
Fetch a CSV example from this website.
q)url:"https://code.kx.com/download/data/example.csv"q)count t:("SFI";enlist csv)0: system "curl -Ls ",url10000q)tid price qty--------------kikb 36.05 90hlfe 96.57 84mcej 91.34 63iemn 57.12 93femn 63.64 54engn 94.56 38edhp 63.31 97ggna 72.39 88mjlg 12.04 58fpjb 34.3 68gfpl 25.34 45jogj 78.67 2gpna 23.08 39njoh 91.46 64aoap 48.38 49bhan 63.2 82enmc 70 40niom 58.92 88nblh 42.9 77jdok 9.42 30..Above, we used system to get a result direct from the operating system, and passed the result to the Load CSV operator 0: to interpret as a table.Table t has 10,000 rows. (The interpreter displayed only the top of the table.)
Save t as example.csv.
q)`:example.csv set t`:example.csvCSVs
Queries¶
If you know SQL you can work with q tables.
q)select from sp where qty>200s p qty---------s1 p1 300s1 p3 400s2 p1 300s2 p2 400s4 p4 300s1 p5 400q)/ double all stocks of p2q)update qty:qty*2 from sp where p=`p2s p qty---------s1 p1 300s1 p2 400s1 p3 400s1 p4 200s4 p5 100s1 p6 100s2 p1 300s2 p2 800s3 p2 400s4 p2 400s4 p4 300s1 p5 400In qSQL queries you can use q operators, keywords, and functions you define yourself.
Language¶
Data¶
Q handles numerical data, including times, dates, and periods.Also booleans, characters, GUIDs, and immutable strings called symbols.
Lists of the same datatype are called vectors and have simple representations.
q)1 2 3*acos -1 / three floats3.141593 6.283185 9.424778q)count 2019.07.05 2019.09.15 2019.11.16 / three dates3q)count 08:30 12:45 17:15 / three times3q)count 22:45:53.600 22:45:53.601 22:45:53.602 / three timestamps3q)count "fox" / three characters3q)count ("quick";"brown";"fox") / three character lists3q)count each ("quick";"brown";"fox")5 5 3q)count `quick`brown`fox / three symbols3q)count each `quick`brown`foxes / symbols are 'atoms'1 1 1Lists¶
Anything can be an item in a list.
q)count (42;"foxes";`screw`bolt;2020.09.15)4In the list above the first and last items, 42 and 2020.09.15, are single values, known as atoms. The other items "foxes" and `screw`bolt are themselves lists.
A list in which all items are atoms of the same datatype is a simple list or vector. Vectors are key to the high performance of kdb+ and have simple representations.
q)count 3 1 4 5 / integers4q)count 3 1 4 5.9 / floats4q)count "jump" / characters4q)"jump"<"n" / four booleans1010bq)count `cow`sheep`cat`dog / four symbols4q)2020.01.01+30 60 90 120 / four dates2020.01.31 2020.03.01 2020.03.31 2020.04.30q)12:00+30 60 90 120 / four times12:30 13:00 13:30 14:00Indexing is zero-origin.
q)"abcdef"[3 4 0 5]"deaf"Brackets are not always needed.Indexing and function application have the same syntax.
q)count[3 1 4 5]4q)count 3 1 4 54q)"abcdef" 5 4 4 3"feed"You can index a table.
q)sp 0 2 / first and third rowss p qty---------s1 p1 300s1 p3 400q)sp `s`p / two columnss1 s1 s1 s1 s4 s1 s2 s2 s3 s4 s4 s1p1 p2 p3 p4 p5 p6 p1 p2 p2 p2 p4 p5q)sp[`qty] / one column300 200 400 200 100 100 300 400 200 200 300 400q)sp[`qty]>200 / flags101000110011bq)where sp[`qty]>200 / indexes0 2 6 7 10 11q)sp where sp[`qty]>200 / indexed rowss p qty---------s1 p1 300s1 p3 400s2 p1 300s2 p2 400s4 p4 300s1 p5 400A table is a list of dictionaries.
Dictionaries¶
Dictionaries are first-class objects.
q)`item`qty`price!(`screw;500;1.95) / record of a saleitem | `screwqty | 500price| 1.95q)pr:`screw`nail`bolt`nut!0.75 3 2.85 0.55 / price dictionaryq)prscrew| 0.75nail | 3bolt | 2.85nut | 0.55q)pr `bolt`nail / indexing2.85 3q)pr>2screw| 0nail | 1bolt | 1nut | 0q)pr*1+.05*pr>2 / 5% increase where price>2screw| 0.75nail | 3.15bolt | 2.9925nut | 0.55A table is a list of dictionaries.
q)sp 0 / first rows | `s$`s1p | `p$`p1qty| 300Joining a dictionary to a table appends a tuple.
q)sp,`s`p`qty!(`s5;`p3;159)s p qty---------s1 p1 300s1 p2 200s1 p3 400s1 p4 200s4 p5 100s1 p6 100s2 p1 300s2 p2 400s3 p2 200s4 p2 200s4 p4 300s1 p5 400s5 p3 159Functions¶
Function notation is simple.A function can have up to eight arguments.Unless explicitly named, the first three arguments are assumed to be x, y, and z.
q){x*x}2 -1.5 174 2.25 289q)el:{[e;a;v;c]"<",e," ",a,"=\"",v,"\">",c,"</",e,">"}q)el["a";"href";"https://example.com/";"link text"]"<a href=\"https://example.com/\">link text</a>"Iteration¶
Control structures such as do and while are rarely used for iteration.
Much iteration is implicit in the operators.
q)2 3 4 + 1012 13 14q)2 3 4 + 10 100 100012 103 1004Most other iteration is handled by keywords, and special operators called iterators.
q)count each ("quick";"brown";"fox")5 5 3q).h.htc[`p;"The quick brown fox"] / mark up"<p>The quick brown fox</p>"q)"The quick brown fox" {.h.htc[y;x]}/ `p`body`html"<html><body><p>The quick brown fox</p></body></html>"q)8 {x,sum -2#x}\1 1 / 10 Fibonacci numbers1 11 1 21 1 2 31 1 2 3 51 1 2 3 5 81 1 2 3 5 8 131 1 2 3 5 8 13 211 1 2 3 5 8 13 21 341 1 2 3 5 8 13 21 34 55Communication¶
Interprocess communication is ‘baked in’ to q. It requires no library code and is easy to set up.
Watch two kdb+ processes communicating through TCP/IP.
Client/server¶
Use two command shells for this.On the left, we have the server task; on the right, the client.
KDB+ 3.7t 2020.01.22 … | KDB+ 3.7t 2020.01.22m64/ 4()core 8192MB … | m64/ 4()core 8192MB … |q)\p 5432 | | q)h:hopen `::5432 | q)h"2+2" | 4 | | q)h "system\"l /Users/sjt/q/sp.q\""q)+`p`city!(`p$`p1`p2`p3`p4`p5 |(`s#+(,`color)!,`s#`blue`green |+`s`p`qty!(`s$`s1`s1`s1`s2`s3` | | q)h "select from sp where s in `s2`s3" | s p qty | --------- | s2 p1 300 | s2 p2 400 | s3 p2 200 | q)On the left, the server task started listening on port 5432. The client task opened a socket to port 5432, getting a handle, which it dubbed h.
The client task sent to the server the expression 2+2 to be evaluated, and received the value 4 in return.
The client task told the server to load the Suppliers and Parts script.The server task session showed that script loaded.The client sent the server a qSQL query and got a table as a result.
Asynchronous calls are only slightly more complicated.
A production system requires code in the callbacks to secure communications but you can see from the above that the basics are very simple.The baked-in interprocess communications make it simple to implement systems as tasks distributed over multiple machines.
Webserver¶
A q session can listen for HTTP requests and act as a webserver.The default callback composes a page for browsing tables in the session.
q)tables[] / Suppliers & Parts`p`s`spq)\p 8090 / listen to port 8090Browse to http://localhost:8090.
Customizing the default webserver
The .h namespace
Development¶
Scripts¶
Write and load scripts to define an application.Scripts are text files.
The sp.q script defines the Suppliers & Parts database and runs some queries on it.
s:([s:`s1`s2`s3`s4`s5] name:`smith`jones`blake`clark`adams; status:20 10 30 20 30; city:`london`paris`paris`london`athens)p:([p:`p1`p2`p3`p4`p5`p6] name:`nut`bolt`screw`screw`cam`cog; color:`red`green`blue`red`blue`red; weight:12 17 17 14 12 19; city:`london`paris`rome`london`paris`london)sp:([] s:`s$`s1`s1`s1`s1`s4`s1`s2`s2`s3`s4`s4`s1; / fkey p:`p$`p1`p2`p3`p4`p5`p6`p1`p2`p2`p2`p4`p5; / fkey qty:300 200 400 200 100 100 300 400 200 200 300 400)select distinct p,s.city from spselect sum qty by p.color from spselect from sp where s.city=p.cityIn scripts, q expressions can be written across multiple lines.
IDE¶
KX Developer is a free interactive development environment (IDE) for q.
