By Peter Simpson, OneTick Product Owner, KX
Following our recent workshop on accessing on-demand market data from KDB-X, I had the opportunity to answer a series of audience questions. While the session was designed for KDB financial professionals interested in simplifying their market data pipelines for quantitative development within KDB-X, the questions we covered apply to users of all analytics platforms.
OneTick Cloud is our high-quality, on-demand managed time-series data and analytics platform that provides users with global, AI-ready market data, instantly seamlessly fueling compute, research, and analytics engines.
Data hydration is the process of taking raw, fragmented, or asynchronous financial data and standardizing, enriching, and cleaning it so it becomes AI-ready. It bridges the gap between raw unstructured data (like raw exchange ticks) and actionable machine learning and quantitative analysis.
The hydration process involves several steps:
For algorithmic trading and AI agents, hydrated data is crucial because it eliminates the "data tax"—the 70% to 80% of time quants typically spend cleaning feeds. By providing fully harmonized, audit-ready time-series data, it allows trading systems to operate without the data gaps that cause hallucinations or inaccurate predictions.
Last week, we discussed the set up and data coverage basics (read here). Today, let's get into the nitty gritty of analytics and querying.
Okay. We saw a few, when I was walking through (watch the recording).
So we are storing data with the exchange symbol, but you can also query, with the Bloomberg symbol, the FIGI, the iSIN, the CDL, or the CUSIP.
If you have a CDL or CUSIP license, then you can query with one symbology and pull back what the matching CDL or CUSIP is.
Yes. We typically retrieve the PCAP files from each venue.
That data gets loaded, both for kind of level one, so trades and quotes and book depth. You can either simply query every underlying kind of order update message, or you can use our OB snapshot or OB summary functions to query the book down to a particular filtered view, which might be by level, by number of shares, by trade value, by spread, by price skew, and that book can be pulled back by book update by book update or periodically as kind of book bars. You're choosing how you want to retrieve the book, and there are, as we saw in the little demo, three different representations of the book.
Or you can use our kind of summary function where you're asking for more analytical questions. So if I wanted to trade a certain size or a certain value, how deep in the book would I need to go, and what's the VWAP bid and ask and the VWAP spread to be able to achieve that trade size.
It's exactly the same process. Issue the SQL, get back the results, and not have to worry about collecting and storing and managing the book data yourself.
Yes. From an access point of view, it's exactly the same as querying for historical data who are just issuing the SQL statements, either for giving me back intraday data for the current time, or you're saying give me back the last price across every instrument within my selected venue.
The only real difference is around exchange permissioning. For you to be able to see a real time venue, your account needs to be permissioned to see that venue in real time, and we need the exchange agreement signed for you to do that.
Okay. Not currently. You're using the kind of OneTick SQL syntax to pull back data.
We're looking at how we can provide queue access later on to the year in the year in our kind of OneTick Cloud environment. So for the moment, from a Q client, you're using the module to issue OneTick SQL and pull back the data into a Q table.
Okay. Now we are storing the data as it's received. We're applying trade corrections or trade cancellations, but it is the data as we receive it.
We also store corporate action adjustment factors and the type of corporate action that's occurred. And we provide functions so that you can query the data unadjusted or query the data after corporate actions have been applied.
And it could be that you just want to query when splits occur or you want to also include dividends. You're choosing how you want to adjust the data based on the corporate actions that you're interested in.
And that's just one additional function into your SQL statements to retrieve the data. And then you could retrieve adjusted or unadjusted and adjusted as additional fields.
Or you could choose just to retrieve the unadjusted and retrieve the adjustment factors and do your adjustment locally. But we're making it so easy. You just pull back the adjusted data.
Yes. So we're storing all of the futures data contract by contract with its history going back.
There may be times where you don't want to look at individual contracts. You want to look at the continuous contract.
For the first twelve months or the first twelve front months, we provide access to continuous contracts based on expiry. So when the contract expires, we'll roll onto the next.
For the front month contract, we also provide alternatives where we pick the contract with the maximum volume or the maximum open interest. So you have a few different ways to query.
When you're querying with a continuous contract, you're using either the exchange code and then a slash and the number. So doing it in the front month or the second month, etcetera.
Or you could also query with the Bloomberg code for the continuous contract as well, either looking at the contracts based on expiry or contracts based on maximum trading volume.
Okay. So especially with equities in various markets around the world, there are multiple venues.
In the case of the US, we have probably twenty different kinds of let venues, and there is the US SIP, so a consolidated tape that's provided, which combines all those venues together, and we provide that in our US comp dataset.
Outside of the US, there is no consolidated tape, so we generate them for you. And we do that for various markets around the world. For certain markets like Australia, India, Japan, there are a few markets.
The same goes for Korea, Taiwan and Mexico. When we're looking at Canada, there are ten to fifteen markets. When we're looking at Europe, we're combining around twenty six different countries and a hundred well, a hundred and fifty plus different mix.
So in these cases where there's not a consolidated tape, we're creating one for you. We're creating our own consolidated BBO. So there's a Canadian BBO. There's a European BBO, Japan, Australia, etcetera.
You can then query this consolidated view of the data and see one place to access every single trade for that particular region and benchmark that trade against the NVBO that we're generating for you. Or you can use our kind of analytical features to create your own NBBO based on the venues that you trade if you want a subset.
Please try it out for yourself. As we saw from the demo, it's a thirty second install, issue the SQL query, and come back with the data. The documentation's on the KX website. Ask us if you have any questions.
OneTick Cloud offers a single vendor end-to-end solution, significantly reducing the time to value. KX and OneTick are the only vendor delivering AI-ready, hydrated, temporal market data as a managed service. Our data is pre-normalized across 250+ venues and 30+ years of history, point-in-time with no look-ahead bias, machine-readable from day one, and fed natively into Python, SQL, and KDB-X.
Save time, money, and resources by letting the KX OneTick team clean feeds, map symbols, and align timestamps so your quants, analysts, and AI models can do their work.
Ready to learn more? Schedule a call with the OneTick team today: contact us here.
Best wishes,
Peter Simpson