Right now the Elastic-Stack (also known as ELK-Stack) is an increasingly used tool to realize BI- and Big-Data projects covering the whole challenges from the ETL-process to the graphical visualization. In one of our projects at the „Digitallab“ OC|Lab we were analyzing Logfiles, trying to improve the readability of the logs and gathering new knowledge within the data. For these tasks we used the powerful Stack.

The Stack is divided in three parts: Logstash, Elasticsearch and Kibana. Briefly summarized Logstash extracts data from various sources, transforms it by defined business rules and sends it to Elasticsearch. The Clusterstorage provides extremely fast and multi-purpose searching. Among others this is possible due to the scalable architecture. Kibana is the frontend of the Stack and is responsible for the visualization of data.

The Elastic website offers complex, in-depth explanations about Scripted Fields. However, the hints Kibana shows while creating a scripted field are very helpful and much easier to understand. An example would be the calculation of the characteristic factor „Sells per [timeunit]“. With this tool you simply have to create a new Scripted Field and write the following Code:


This script now calculates the sells per minute if needed. Of course you can do more challenging operations. For those you will need the language „painless“ (same Syntax as Groovy) in order to access multiple fields and apply logic on top. Furthermore you can import plugins to create Scripted Fields with JavaScript or Python. Another sample:


If you“™re interested in learning more about Scripted Fields: https://www.elastic.co/guide/en/elasticsearch/reference/5.2/modules-scripting.html

Alle Beiträge von danielzellmann

Schreibe einen Kommentar