Development of Efficient Connector Engine for Effective Insights

Technology

Development of Efficient Connector Engine for Effective Insights

06

Engineers

04

Months

90%

Reduction in Operation Cost

Overview

TechVariable developed a connector engine, making it possible to extract customizable user-defined data from any data source. The connector supports the OData sources, including Microsoft Azure, SharePoint, Office 365, Salesforce, and Tableau. It also allows data extraction from non-OData sources like JIRA, REST, and SOAP APIs. The extracted data is then brought to the connector/adapter store based on the user inputs.

Services

Product Engineering

Technology

JAVA Springboot, Grakn, Apache ODATA Olingo library, Angular, Mongo DB, Node.js

Location

Malaysia

Challenges

The client is an HR tech giant that provides its customers with Key Performance Indicators (KPIs). Their customers used disparate, unconnected off-the-shelf applications for various business functions. It prevented them from generating meaningful, organization-wide KPIs. Difficulties in curating the data from such disparate sources hampered any Business Intelligence (BI) operations challenging.

Solutions

The connector would fetch details of available data based on the application chosen by the user. The user can then select the data types required for analysis. The connector then injects the data into the BI pipeline to extract valuable insights in the form of KPIs.

Modules implemented

Data Publishing for Downstream System

The connector pushes extracted data to the downstream BI pipeline for further processing, analysis, and insights generation.

AI-based Schema Generation

Using Grakn, the connector generates the schema using the metadata specified by the user.

Meta Data Extraction Module

Once the user defines the data source for extraction, the connector extract metadata. Users can select the exact information they require from this metadata.

Scheduler or Incremental Fetch

A schedule can be set, and the connector extracts the data automatically at predefined periods.

High Level Design Architecture

Need to estimate for your next project?

We at TechVariable do acknowledge that one size will not fit all. Hence, we work in collaboration with you to identify, analyze & then develop a solution that fulfills your needs. Either we will define the functional scope of your project to estimate the timeline and budget or you can create your own agile team from among our resources.