Populating Demyst Catalog in third-party applications

Let’s say you have an internal data catalog but you want to populate it with external data catalog to increase the breadth of datasets available in your organization or just educate your staff on what’s available out there. Your one-stop shop for this is Demyst and in this post, we will see how to populate data.world with our product catalogs using our favorite tool, python. 

Before we get started, you would need to sign-up on Demyst and Data.world. It took me a minute to sign-up and update my profile on these websites.

The first thing we are going to do is see a sample of product catalog available on Demyst and what will be populated into the data.world. To do this, we are using our python API and calling our, you guessed it, product_catalog() method. You can find more information on our catalog methods and data dictionaries over here.

demyst product catalog

Heading over to data.world, you will need to enable Python as a valid integration and create an API token. And just like our python API, installing it and configuring to use the API token was super easy and each a one-liner. 

We now create a dataset using a sample CSV file that gives us the data.world URL for our dataset. 

create dataset

We can now add each product from Demyst as a separate CSV file or upload all the products catalog in a single master CSV file in this dataset.  And these few lines can be converted to a script that a user can run one time or periodically to add all metadata from the Demyst product catalog into their own catalog.

adding files to dataset

Let’s see the dataset we just created

load datset

We now have the complete catalog as an external dataset available to us. This becomes beneficial to us for investigating the products in an environment we have already integrated with and sharing it across our organization. But let’s not stop at ingesting the catalog and create a project associated with this dataset.

creating project

Creating a project linked with our dataset gives us the opportunity to run queries and create insights that can further be shared and discussed. So let’s create a sample query that finds us all the NAICS attribute and the products from Demyst that provide this attribute in their response.

query dataset

Lastly, let’s see that query as an insight saved to our project with some visualization using Chart-Builder that provides vega-lite visualizations. This helps in seeing how many provide NAICS codes and how many of those are NAICS description. The visualization might be lite but don’t let us inhibit you from more advanced visualizations.


That was pretty simple, wasn’t it? And you have the ability to rerun the selected few lines to update your dataset periodically and create complex insights into hundreds of attributes from Demyst. You can try these steps on our hosted notebook and contact us at client@demystdata.com to let us know about it.



Harshit Singh

Harshit Singh

Share on facebook
Share on twitter
Share on linkedin

More to explore

Socialgist – Data Spotlight

Are you on the hunt for accessible social data? Do you need access to the world’s public online conversational data? We are excited to announce that Socialgist is the latest addition to the Demyst platform.

Read More »
Close Menu