Creating unparalleled data quality in preparation for Gen AI

FST QLD 2024: Roundtable Recap & Key Takeaways

The FSTGov Future of Queensland Government Summit returned for its eighth annual edition in March, uniting over 120 executives, thought leaders and technologists to explore Queensland’s digital innovation roadmap.

After a terrific summit discussing Queensland’s thriving digital future, I wanted to put together some key takeaways from the dynamic roundtable discussions I hosted on behalf of EncompaaS, centered on creating a foundation of unparalleled data quality in preparation for Generative AI.

One of the key themes during the conference was how technology can empower us to provide superior services and more responsive solutions for citizens. So, during my roundtable discussions, we talked about how government departments can effectively leverage AI to deliver better customer service, the benefits of a single customer view for all enterprise content, and how to approach the barriers to AI adoption.

Here are the key questions and takeaways that came out of these discussions:

What are you using (or not using) Generative AI for?

• Government employees are increasingly leveraging Large Language Models (LLMs) for tasks ranging from summarising content, asking questions of bodies of text, rewriting content and even in some cases, code generation.
• QChat, a LLM based service for Queensland Government, came up in a lot of the discussions – particularly how people are currently using it to ask questions about organisational data, and to provide a body of text and ask questions about that text.
• Some use of public ChatGPT, with mandated guidelines prohibiting submission of sensitive data, is also in use under a self-policed system.
• There was also some early use of Microsoft Copilot among attendees, but inconsistent results have not instilled confidence yet.

What are the concerns or challenges you have with adopting Generative AI? 

• Concerns surrounding the adoption of Generative AI persist among stakeholders. Some attendees I spoke to said that queries being submitted are used to contribute to the model being used and therefore, accidentally making sensitive data available through reverse engineering of the model.
• Another common concern is hallucinations, resulting from poor data being used for the basis of the result. So too is lineage and the ability to understand what information was used to provide the answer.
• Attendees also raised concerns around how to secure data in a manner that assures sensitive data is not included in results to maintain information security.
• Additionally, supervising results so that false positives don’t become fact and lead to unintended outcomes was also noted. 

How do you ensure that the right information is provided to avoid hallucinations?

• To mitigate the risk of erroneous outputs and hallucinations, some organisations are adopting proactive measures such as manually assembling curated datasets that they know to be correct and only using this data for generative queries
• In addition, there was broad consensus on the importance of consistent, normalised data to underpin the efficacy of AI and in particular, Generative AI applications.

It was great to hear the differing perspectives on Gen-AI first-hand and discuss how vital intelligent information management is in enabling government departments to innovate and take advantage of these new technologies. While there are a diverse and valuable set of vendors out there who can work alongside government departments, what resonated well with the attendees I spoke to about the EncompaaS solution was our ability to find, enrich, organise, and de-risk data across the enterprise.

Essentially, by helping you understand your information you can access the right information to help answer your questions. In a practical sense, this could include doing a capability report on hiring for the last five years and using EncompaaS to provide the employment contracts for new hires during this time frame to answer this query accurately. Similarly, you could be doing an RFP for building a path and can easily find all other RFPs created for similar projects.

What also resonated was EncompaaS’ ability, when presenting data for Generative AI, to exclude data that the requesting person had no permission to, as well as actively excluding sensitive data. This was a lightbulb moment for many attendees, as it provides a secure solution now that LLMs simply don’t have.