High-Performance 2D/3D Visualization of Massive Datasets

Data Science isn't just about finding ready-made applications online and trying to adapt them to your needs, of course. What makes the difference is performing exploratory data analysis, applying statistical tests, and extracting insights. Data visualization is an integral part of this exploration. There are many libraries available for data visualization, but what if your dataset contains millions of samples? Ever had graphics start loading slowly, and navigating within them become nearly impossible? I certainly have. :) Below, I'm sharing two example applications—in both 2D and 3D—using the Datoviz library, which visualizes data by leveraging your computer's GPU resources through Vulkan, allowing us to navigate smoothly through millions of data points.

First of all, we need to install Datoviz.

pip install datoviz

2D Example

Data reading is considered as an abstract concept rather than a concrete implementation. Details are given in the code as comment.

And here’s the output! :) We’re working with thousands of data points, which we can navigate through smoothly. Even millions of data points aren’t a problem—you’ll see that in the 3D section. However, visualizing time series with millions of data points can be challenging, which is why I prefer using thousands instead.

Please wait while the visual loads. :)

3D Example

Again, data reading is considered as an abstract concept rather than a concrete implementation. Details are given in the code as comment.

Now we have millions of datapoints. :) Here you go!

Please wait while the visual loads. :)

As you can see, with just a few lines of code, you can smoothly navigate through your data by leveraging GPU acceleration via Vulkan.

Special thanks to Cyrille Rossant and everyone who contributes to the development of Datoviz [Web | GitHub].

Enjoy! See you at the next post. :)