Scripted Fields in the Elastic-Stack

Right now the Elastic-Stack (also known as ELK-Stack) is an increasingly used tool to realize BI- and Big-Data projects covering the whole challenges from the ETL-process to the graphical visualization. In one of our projects at the “Digitallab” OC|Lab we were analyzing Logfiles, trying to improve the readability of the logs and gathering new knowledge within the data. For these tasks we used the powerful Stack.

The Stack is divided in three parts: Logstash, Elasticsearch and Kibana. Briefly summarized Logstash extracts data from various sources, transforms it by defined business rules and sends it to Elasticsearch. The Clusterstorage provides extremely fast and multi-purpose searching. Among others this is possible due to the scalable architecture. Kibana is the frontend of the Stack and is responsible for the visualization of data.

The Elastic website offers complex, in-depth explanations about Scripted Fields. However, the hints Kibana shows while creating a scripted field are very helpful and much easier to understand. An example would be the calculation of the characteristic factor “Sells per [timeunit]”. With this tool you simply have to create a new Scripted Field and write the following Code:

scriptedfields1

This script now calculates the sells per minute if needed. Of course you can do more challenging operations. For those you will need the language “painless” (same Syntax as Groovy) in order to access multiple fields and apply logic on top. Furthermore you can import plugins to create Scripted Fields with JavaScript or Python. Another sample:

scriptedfields2

If you’re interested in learning more about Scripted Fields: https://www.elastic.co/guide/en/elasticsearch/reference/5.2/modules-scripting.html

Dieser Beitrag wurde unter Analytics & BigData abgelegt und mit , , verschlagwortet. Setze ein Lesezeichen auf den Permalink.

Kommentar verfassen

Trage deine Daten unten ein oder klicke ein Icon um dich einzuloggen:

WordPress.com-Logo

Du kommentierst mit Deinem WordPress.com-Konto. Abmelden / Ändern )

Twitter-Bild

Du kommentierst mit Deinem Twitter-Konto. Abmelden / Ändern )

Facebook-Foto

Du kommentierst mit Deinem Facebook-Konto. Abmelden / Ändern )

Google+ Foto

Du kommentierst mit Deinem Google+-Konto. Abmelden / Ändern )

Verbinde mit %s