Using SQL Gateway with Python, Vector Search, and Interoperability in InterSystems Iris

Part 3 – REST and Interoperability

Now that we have finished the configuration of the SQL Gateway and we have been able to access the data from the external database via python, and we have set up our vectorized base, we can perform some queries. For this in this part of the article we will use an application developed with CSP, HTML and Javascript that will access an integration in Iris, which then performs the search for data similarity, sends it to LLM and finally returns the generated SQL. The CSP page calls an API in Iris that receives the data to be used in the query, calling the integration. For more information about REST in the Iris see the documentation available at https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cls...

Hey, community! 👋

We are a team of Stanford students applying technology to make sense of climate action. AI excites us because we know we can quickly analyze huge amounts of text.

As we require more reports on sustainability, such as responsibility reports and financial statements, it can be challenging to cut through the noise of aspirations and get to the real action: what are companies doing

That’s why we built a tool to match companies with climate actions scraped from company sustainability reports.

Using SQL Gateway with Python, Vector Search, and Interoperability in InterSystems Iris

Part 2 – Python and Vector Search

Since we have access to the data from our external table, we can use everything that Iris has to offer with this data. Let's, for example, read the data from our external table and generate a polynomial regression with it.

For more information on using python with Iris, see the documentation available at https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cls?KEY=AFL_epython

Let's now consume the data from the external database to calculate a polynomial regression. To do this, we will use a python code to run a SQL that will read our MySQL table and turn it into a pandas dataframe:

You need to install the application first. If not installed, please refer to the previous article

Application demonstration

After successfully running the iris image vector search application, some data needs to be stored to support image retrieval as it is not initialized in the library.

An experiment on how to use the LangChain framework, IRIS Vector Search, and LLMs to generate IRIS-compatible SQL from user prompts.

This article was based in this notebook. You can run it with a ready to use environment with this application in OpenExchange.

Setup

First, we need to install the necessary libraries:

I recently had to refresh my knowledge of the HealthShare EMPI module and since I've been tinkering with IRIS's vector storage and search functionalities for a while I just had to add 1 + 1.

For those of you who are not familiar with EMPI functionality here's a little introduction:

Enterprise Master Patient Index

In general, all EMPIs work in a very similar way, ingesting information, normalizing it and comparing it with the data already present in their system. Well, in the case of HealthShare's EMPI, this process is known as NICE:

🌍 Inclusion & Innovation in Education 🌍
Our project reimagines learning for all students, with a focus on accessibility and interactive experiences. Built with the goal of making education engaging and inclusive, the tool is designed to support students of all abilities in learning complex material in an intuitive way.

💡 What It Does
This educational app transforms lesson presentations into interactive study sessions:

In ObjectScript you have a wide collection of functions that return some value
typically:

set variable = $somefunction(param1,param2, ...)

There is nothing special about that.
But there is a set of functions that I classify as LEFT SIDED
The specialty of them is that you can use them also on the left of the equal operator
as a target in the SET command:

set $somefunction(param1,param2, ...) = value

In the business world, every second counts, and having high-performing applications is essential for streamlining our business processes. We understand the significance of crafting efficient algorithms, measurable through the big O notation.

We've recently made available a new version of InterSystems IRIS in the Vector Search Early Access Program, featuring a new Approximate Nearest Neighbor index based upon the Hierarchical Navigable Small World (HNSW) indexing algorithm.

Hi, Community!

Want to dive deeper into the concepts of vector searches and embeddings? Learn from an InterSystems expert:

What Are Embeddings?

https://www.youtube.com/embed/WAKwI-ijoQE?utm_source=youtube&utm_medium=social&utm_campaign=WAKwI-ijoQE
[This is an embedded link, but you cannot view embedded content directly on the site because you have declined the cookies necessary to access it. To view embedded content, you would need to accept all cookies in your Cookies Settings]

Hi Community,

Play the new video on InterSystems Developers YouTube:

⏯ InterSystems IRIS Vector Search and the Python Ecosystem @ Global Summit 2024

https://www.youtube.com/embed/XVEvUOxJUrw
[This is an embedded link, but you cannot view embedded content directly on the site because you have declined the cookies necessary to access it. To view embedded content, you would need to accept all cookies in your Cookies Settings]

There are a lot of great community articles regarding "vector search on IRIS", and samples in OpenExchange. Everytime I see these, I'm so excited to know that so many developers already try vectors on IRIS!

But if you've not tried "Vector Search on IRIS" yet, please give me one minute 😄 I create one IRIS class - and with only one IRIS class you can see how you put vector data in your IRIS database and how you compare these in your application.

Artificial intelligence (AI) has transformative potential for driving value and insights from data. As we progress toward a world where nearly every application will be AI-driven, developers building those applications will need the right tools to create experiences from these applications. Tools like vector search are essential for enabling efficient and accurate retrieval of relevant information from massive datasets when working with large language models.