Observability practitioners may be familiar with using one query language for logs, another for metrics, and another for traces and monitoring application performance. This can sometimes pose challenges when it comes to bringing data together from different datasets and doing it all in one query language.
Elastic NV hopes to solve this problem by launching the Elasticsearch query language. It’s a pipelined query language, according to Ken Exner (pictured), chief product officer at Elasticsearch Inc.
“[It] “It allows developers and practitioners to query different datasets, bring data together in one query language, be able to quickly create fields, be able to do math and operations, and do joins on different datasets,” Exner said. “It’s a very powerful new query language that allows people to do things in one query language that were only possible in different query languages.”
Exner spoke with CUBE industry analyst Rebecca Knight at the “Supercloud 5: The Battle for AI Supremacy” event live on theCUBE, SiliconANGLE Media’s live streaming studio. They discussed the problems that Elasticsearch Query Language aims to solve and how the new language relates to artificial intelligence.
Pull data together
Observability practitioners, like SRE practitioners, have always had difficulty bringing data together from different datasets and implementing it in a single query language, according to Exner. The goal of Elasticsearch Query Language is to enable customers to have a single query language that allows them to work across different data sets, bring things together and do things they couldn’t do before.
“Because it’s a new query engine built directly into Elasticsearch, it’s native to the engine, and it’s extremely fast. Customers have always loved the speed of Elasticsearch in searching.” Now, with this new query API and new query engine, we have extremely fast query performance as well “
Since it’s 2023, all the hype is about generative AI. But the thing about generational AI is that it’s only as good as the foundation it’s built on, according to Exner.
“When you use code generation, for example, you need to build on top of APIs, on top of libraries, on top of frameworks. If you’re using a code generation tool and you say you’re trying to build a microimage service on top of an object storage system, like something creates thumbnails, the PC won’t be built first.” S3 won’t be built first. It will start from some foundational primitives and then build on them.
Flexible AI Assistant
When considering a thumbnail service, it will build on S3 libraries that are built on top of S3, according to Exner. If a client wants to perform some analysis on different datasets, they want to build it on top of some basic fundamentals.
“That’s what ESQL gives them. It gives them a really strong foundational language for how to pull data from different data sets, whether it’s structured data or unstructured data, and be able to do different types of aggregations, set alerts based on whatever kind of setup you want to come up with.” “Language, creating detection rules that work in real time,” he said. “All of these things can happen in addition to the data you bring in.”
Exner added that if customers want to build a natural language interface on top of that, the company can do that as well. This is where the company’s flexible AI assistant comes into play.
“The AI Assistant allows you to use natural language to map into fluid query language. So, you can just think about how you want to build a query.”
One might want to pull data from a data source and look at all hosts that have a CPU above 40% and that have a certain type of latency, for example, greater than 2 seconds. They may want to look at places where errors occurred by more than 50% in the last hour.
“Building a query like this across different datasets and executing it in natural language is now possible because of the core capabilities we have with ESQL,” Exner said.
Here’s the full video interview, part of coverage by SiliconANGLE and theCUBEE from “Supercloud 5: Battle for AI Supremacy” event.:
(*Disclosure: This portion of theCUBE was sponsored by Elasticsearch Inc.. Neither Elasticsearch nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)
Photo: Silicone Angel
Your upvote is important to us and helps us keep the content free.
One click below supports our mission of providing free, deep, relevant content.
Join our community on YouTube
Join a community of over 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies Founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more notable figures and experts.
Thank you
In the ever-evolving world of data management and analytics, the concept of elasticity has become increasingly important. By leveraging a custom query language, elasticity aims to achieve faster query performance and improved scalability for data processing. This approach enables organizations to efficiently handle large and complex datasets, thereby empowering them to derive valuable insights and make data-driven decisions in a timely manner. With the ability to adapt to changing workloads and data volumes, elasticity holds the potential to revolutionize the way businesses approach query performance and data processing.