Logstash: How to check if a field exists?

 

TL;DR: 

 if [yourField] or [yourField] =~ /.*/ {

    # Your code

 }

Why?

In Logstash [yourField] could come in one of these datatypes:

• [yourField] = String
• [yourField] = Null
• [yourField] = Number
• [yourField] = Boolean

i.e. [level] = "ERROR"

And also could have sub-fields and therefore these combinations:

• [yourField][yourSub] = String
• [yourField][yourSub] = Null
• [yourField][yourSub] = Number
• [yourField][yourSub] = Boolean

i.e. [request][time_milliseconds] = 73

If we use "if [yourField]" to check if the field exists, turns out that not always matches and if we use "if [yourField] =~ /.*/" not always matches.

Result:

	if [yourField]	if [yourField] =~ /.*/
[yourField] = "foo"	Match	Match
[yourField] = ""	NO	Match
[yourField] = 0	Match	NO
[yourField] = True	Match	NO
[yourField][yourSub] = "foo"	Match	NO
[yourField][yourSub] = ""	Match	NO
[yourField][yourSub] = 0	Match	NO
[yourField][yourSub] = True	Match	NO

• "if [yourField]" won't work when the field exists but is Null.
• "if [yourField] =~ /.*/" won't work when is a number, boolean or has sub-fields.

The safer approach is:

if [yourField] or [yourField] =~ /.*/ {

    # Your code

}