Enhanced Document Chunking

Enhanced Document Chunking

February 7, 2023

February 7, 2023

February 7, 2023

February 7, 2023

v 1.2.1

Remember our document serialization machine? It chunks, queues, and passes documents into our inference pipeline, for every new document everyday. Well it wasn't very performant after we put the first version into production so we had to crank it up a notch. We made some massive improvements to its distributive qualities by introducing Dask to the document chunking.

Remember our document serialization machine? It chunks, queues, and passes documents into our inference pipeline, for every new document everyday. Well it wasn't very performant after we put the first version into production so we had to crank it up a notch. We made some massive improvements to its distributive qualities by introducing Dask to the document chunking.

Remember our document serialization machine? It chunks, queues, and passes documents into our inference pipeline, for every new document everyday. Well it wasn't very performant after we put the first version into production so we had to crank it up a notch. We made some massive improvements to its distributive qualities by introducing Dask to the document chunking.

Remember our document serialization machine? It chunks, queues, and passes documents into our inference pipeline, for every new document everyday. Well it wasn't very performant after we put the first version into production so we had to crank it up a notch. We made some massive improvements to its distributive qualities by introducing Dask to the document chunking.