Dashboard Web Interface

This blog will explore the process of deploying a simple analytics pipeline where we will present the data parsed and analyzed in a jupyter notebook in a web front-end hosted on AWS Amplify. We are using the ads impression and costing database for our model, but the pipeline architecture should work for any raw data.

I am building a data streaming platform for a visual effects production company. This article will follow my current architecture for the platform.

Goal : Visualize the render jobs being spooled on our render farm to analyze which shows are taking up the most resources and allocate them appropriately.

This architecture can be divided into 4 key parts,

  1. Capture raw sensor data stream.
  2. Source and Process the data stream in Apache Kafka.
  3. Store the stream in a timeseries database.
  4. Visualize the stored stream in a dashboard.

All our application containers will be deployed and orchestrated using Kubernetes for scalability and…

I work for a VFX House as a Software Developer. In entertainment industries, our data is our valuable Intellectual Property. So machines in our architecture are never fully connected to Internet to maintain a secure platform. In such cases, bootstrapping kubernetes can be a workaround as many components assume the internet connectivity and make various API calls to bring in images and configurations.

Hopefully this will help you build up your kubernetes platform fast and save you from hefty amount of googling. …

We live in the age of compression.

Everything is getting compressed and redefined everyday, whether its a phone, a television set or data itself for fast travel on the internet.

What Happened Last Month video series aims to compress events happened over a month into 2–5 min videos.

This representation of video data imparts a lot of information in short span of time, which is a need to enlighten the Millennial's and Gen Z demographics on subjects which make a difference in the world.

We covered varying topics. From cultural happenings, political news to sports.

Hopefully the reader took the notice of the video and has critical comments which are extremely valuable to us content creators.


Harsh Dhillon

I think therefore I am.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store