Development of Efficient Connector Engine for Effective Insights

Technology

Development of Efficient Connector Engine for Effective Insights

06

Engineers

04

Months

90%

Reduction in Operation Cost

Overview

TechVariable developed a connector engine, making it possible to extract customizable user-defined data from any data source. The connector supports the OData sources, including Microsoft Azure, SharePoint, Office 365, Salesforce, and Tableau. It also allows data extraction from non-OData sources like JIRA, REST, and SOAP APIs. The extracted data is then brought to the connector/adapter store based on the user inputs.

Services

Product Engineering

Technology

JAVA Springboot, Grakn, Apache ODATA Olingo library, Angular, Mongo DB, Node.js

Location

Malaysia

Challenges

The client is an HR tech giant that provides its customers with Key Performance Indicators (KPIs). Their customers used disparate, unconnected off-the-shelf applications for various business functions. It prevented them from generating meaningful, organization-wide KPIs. Difficulties in curating the data from such disparate sources hampered any Business Intelligence (BI) operations challenging.

Solutions

The connector would fetch details of available data based on the application chosen by the user. The user can then select the data types required for analysis. The connector then injects the data into the BI pipeline to extract valuable insights in the form of KPIs.

Modules implemented

Data Publishing for Downstream System

AI-based Schema Generation

Meta Data Extraction Module

Scheduler or Incremental Fetch

Data Publishing for Downstream System

The connector pushes extracted data to the downstream BI pipeline for further processing, analysis, and insights generation.

AI-based Schema Generation

Using Grakn, the connector generates the schema using the metadata specified by the user.

Meta Data Extraction Module

Once the user defines the data source for extraction, the connector extract metadata. Users can select the exact information they require from this metadata.

Scheduler or Incremental Fetch

A schedule can be set, and the connector extracts the data automatically at predefined periods.

High Level Design Architecture

Need to estimate for your next project?

We at TechVariable do acknowledge that one size will not fit all. Hence, we work in collaboration with you to identify, analyze & then develop a solution that fulfills your needs. Either we will define the functional scope of your project to estimate the timeline and budget or you can create your own agile team from among our resources.

The Result

1. The customers can now pull data from multiple, disjointed data sources to generate meaningful, holistic insights and KPIs.

2. The automation through schedulers and data push to BI systems increases the efficiency of the KPI generation.

Previous slide

Next slide

See how our solutions are making a difference in healthcare

E-commerce Portal with Product Traceability

TechVariable developed an agri-tech online platform with an e-commerce portal using the Hyperledger Blockchain framework for traceability and management in agri-business, ensuring transparency and building trust between customers and farmers.

70%

Increase in operational efficiency

03 Engineers

05 Months

Facilitating Anthropological Survey Through Web Portal and Mobile Application

We created a web portal and mobile app for Gauhati University’s Anthropology Department to digitize their paper-based survey process. The app supports offline mode, captures geolocation, and syncs data when internet is available. The portal provides secure login access to extract and sort/filter results.

30%

Increase in Data Accuracy

03 Engineers

04 Months

TechVariable develops modular information aggregation platform for Director Intel

TechVariable built a scalable, reliable platform for Director Intel to aggregate real-time and accurate information on the Russell 3000 Index. Using a microservice architecture, NLP algorithms, and third-party services, the platform offers easy indexing, sharable resources, and visually understandable data.