Streamlining KOL Identification: A Journey Towards AI-Driven Healthcare Data Solutions

Healthcare

Streamlining KOL Identification: A Journey Towards AI-Driven Healthcare Data Solutions

04 Engineers

02 Months

60%

Efficiency in Search Function

Technology

Python, Javascript, Flask, React, GraphQL

Services

Rapid Application Development

Location

Seattle, USA

Overview

A Seattle-based tech startup sought a solution to streamline the time-consuming task of manually searching and aggregating Key Opinion Leaders (KOL) data in the healthcare sector. The client aimed to integrate information from multiple sources into a single platform, presented to end users through robust visualizations like knowledge graphs. We were tasked to develop modules for the product’s new feature, covering UX/UI, front-end, and back-end aspects.

Challenges

Time Constraint and Learning Curve : Working in a different time zone and within a tight one-month timeframe, the engineers had to quickly adopt new coding languages and techniques.
Data Integration: Efficient incorporation of numerous JSON files, including data on healthcare practitioners, organizations, open payments, and taxonomy, into Neo4j, a graph database, was required.
Analytical Graphs: Real-time aggregation queries on millions of Neo4j data sets were necessary to create dashboard analytical graphs.
Global Search AutoComplete: A feature enabling users to input any character and quickly retrieve data where any node's property contained those keywords were needed.
Custom Knowledge Graphs: The project demanded the creation of various knowledge graphs with different custom designs.

High Level Design Architecture

Solutions

Efficient Data Conversion: The team converted JSON files into CSV format, creating separate CSV files for nodes and relationship data. Using a Neo4j admin tool to import the CSV files expedited the insertion process, managing approximately 70 million nodes and 200 million pieces of relationship data in under six minutes.
Precalculated Aggregations: The team precalculated expensive aggregations and stored the data on new nodes and relationships, given that graph-generating queries were primarily OLAP.
Full-Text Search Implementation: A full-text search on nodes was enabled using Lucene indices and a distance-similarity calculation, ensuring quick results.
Streamlined Knowledge Graph Generation: A single component for tables, line charts, and bar charts was used, with the team dynamically passing the designs attached to each graph type to the desired components.

Result

The Global Search Autocomplete feature enabled users to search millions of KOL data, publications, clinical trials, and industry-related information simultaneously. The team also linked healthcare practitioners’ Twitter accounts to search results, providing end-users with real-time trends and similar Twitter accounts in the therapeutic field. This new, cost-effective search tool with unique modules gave the client a competitive edge in the industry.