I have performed a number of updates to the PowerShell SYSLOG module since this post. You can read the latest post. The module has been renamed to Posh-SYSLOG.

The GitHub location has been moved to https://github.com/poshsecurity/Posh-SYSLOG.

The module is now available on the PowerShell Gallery.

I often wonder, as I am sure most developers do, if people ever actually read and use the code that I post online. Was it helpful to them or was it useless? Did they use it for something interesting? One piece of code which I know people do use, is my PowerShell SYSLOG code.

A few weeks ago, a user opened my very first GitHub issue! This issue appeared at first to be simple, but as the user and I started to delve into the complexities of the various SYSLOG RFCs, I realized it was far from it.

Before we get into the issue, the code and the resolution, it is worth highlighting that there quite a few IETF RFCs that relate to SYSLOG messages. The two primary ones being:

• RFC 3164 - BSD SYSLOG. This actually wasn't an IETF standard.
• RFC 5424 - IETF SYSLOG. This is a IETF standard. This obsoletes RFC 3164.

There are also RFCs:

• RFC 3195 - Reliable Delivery for SYSLOG
• RFC 5425 - TLS Transport Mapping for 
• RFC 5426 - Transmission of SYSLOG Messages over UDP
• RFC 5427 - Textual Conventions for Syslog Management
• RFC 5848 - Signed Syslog Messages
• RFC 6012 - Datagram Transport Layer Security (DTLS) Transport Mapping for SYSLOG
• RFC 6587 - Transmission of SYSLOG Messages over TCP

I want to send a very big thanks out to the user, DFCH for reporting the issue, helping me understand the RFCs in question and also testing the resulting code.

The Issue

So what was the issue? As DFCH stated:

I will admit that I hadn't ready RFC 5424 or RFC 3164 in a huge amount of detail. As soon as I did it was very obvious that the code was not producing an appropriate timestamp, it also become evident that the overall message I was sending did not meet the RFC specification.

From my analysis, it appeared that I had crossed parts of both RFC 5424 and RFC 3164, ending up with code that wasn't fully complaint to either, and in the long run, not entirely useful.

As DFCH reported, the code didn't not generate the appropriate timestamp, with issues in how it was formatted as well as the precision. Resolving these issues was quite simple, the timestamp could be formatted as recommended by DFCH, this not only resolved the format issue but also increased the precision. Validation of the caller specified timestamp was also easy to implement. I simply changed the parameter to take an object of type DateTime instead of a String. 

But this was just the start of the fixes, as I continued to read and understand the RFCs, I realized my messages were incorrectly formatted as well.

Message Formats

There are two valid SYSLOG message structures as defined in RFC 3164 and 5424. 

Firstly, RFC 3164 specifies the message structure to be the following:

<PRI>TIMESTAMP HOSTNAME TAG CONTENT

Where:

• PRI - Value based on severity and facility
• TIMESTAMP - What date and time with format MMM dd HH:mm:ss
• HOSTNAME - Who is sending the message
• TAG - Name of the process or program generating the message
• CONTENT - Obviously the message being sent

Next, RFC 5424 specifies the message structure as:

<PRI>VERSION TIMESTAMP HOSTNAME APPNAME PROCID MSGID STRUCTUREDDATA [CONTENT]

Where:

• PRI - Value based on severity and facility
• VERSION - Version of the SYSLOG message (typically 1)
• TIMESTAMP - What date and time with format: yyyy-MM-ddtHH:mm:ss.ffffffzzz
• HOSTNAME - Who is sending the message
• APPNAME - Name of the process or program generating the message
• PROCID - Process ID of the application or script
• MSGID - An Identifier to assist in troubleshooting
• STRUCTUREDDATA - RFC 5424 specifies a method of sending key/value pairs
• CONTENT - Obviously the content of the message

One thing to note with RFC 5424 is that the majority of the fields are optional, you still need to send something to ensure the correct layout however, so the nil value "-" is sent. I should also point out that the RFC states that the CONTENT section at the end is completely optional. If nothing is sent, you don't need to even send the nil value. 

My original code on the other hands, was sending messages with the structure of:

<PRI>TIMESTAMP HOSTNAME CONTENT

Where:

• PRI - Value based on severity and facility
• TIMESTAMP - What date and time with format: yyyy:MM:dd:-HH:mm:ss zzz
• HOSTNAME - Who is sending the message
• CONTENT - Obviously the message being sent

How did this happen? 

Well, there are a few reasons why this occurred, in no particular order.

1. I borrowed some of the logic and ideas from other .Net and PowerShell code samples
2. I didn't read either RFC
3. The SYSLOG servers I tested against were not stringent in their rendering of messages received.
4. I referred to Wikipedia when I was checking that my messages were correctly formatted.

In hindsight, the biggest mistakes were using Wikipedia as my guide, and not testing against a more RFC compliant server.

Using Wikipedia as a source for developing compliant code is probably a bad idea, on this occasion it was a great learning experience. Previously the SYSLOG Wikipedia article did not correctly describe the layout and formatting of the full message, crucially missing out the information about the TAG field. The article has been updated since then with corrections ensuring that it is more understandable. It should be noted however that overall, the Wikipedia article is still focused on RFC 3164 and not 5424.

Let’s look at how we clean-up the code.

Additional Parameters

To ensure that we have enough information to support RFC 5424, I needed to add some additional Parameters (which are not mandatory). These include ApplicationName, ProcessID, MessageID, StructuredData and a switch RFC 3164.

The switch RFC 3164 will simply tell the code to send a message in the RFC 3164 format, instead of sending it via RFC 5424 which is its default.

Hostname Generation

Previously, if the hostname parameter was not specified in my code, I simply used the hostname.exe. Whilst there isn't any problems with this, however RFC 5424 actually specifies in some detail how the hostname field should be determined:

1. The FQDN of the server
2. A static IP address
3. The hostname of the server (Windows will always have on of these)
4. Dynamic IP address
5. A NILVALUE (-)

I have updated my code to generate the hostname component of the SYSLOG message via the first 3 steps.

Application Name

The Application name can be a little difficult. Typically from within a function we can determine the name of the script which is calling the function via 2 properties of the $myInvocation variable: ScriptName and PSCommandPath. I have used ScriptName with success in the past, and hence decided to use it again. There is one thing to note, if I am sitting at a console and call send-syslogmessage, then ScriptName will be null, and if that is the case, we will simply use “PowerShell”.

Process ID 

The Process ID is new requirement to ensure RFC 5424. We get this simply from the $PID global variable.

Message ID and Structured Data

These two will always be user specified, if the user doesn't specify them, then send the default RFC 5424 nil value of "-".

Message Generation

Now that we have all of the information required for either RFC, we can now look at message generation. When it comes time, I simply have an ‘if’ statement that controls which format we want to use. The script will then format the timestamp and message accordingly.

Message generation looks like the following:

For RFC 3164, I fixed up the timestamp, and also added in the application name. For RFC 5424 there are some significant changes. I now correctly include the SYSLOG version (1), and then included the corrected timestamp, application name, process ID, message ID and structured data.

The Future

If there was a demand, I would be interested in extending the CMDLet to support the transmission of messages via TCP, as well as sending signed messages. Right now, I don't have a need for such things.

Conclusion

Now that all of those changes have been completed and tested, I have pushed the changes up to the PowerShellSyslog GitHub repository.

I want to thank DFCH again for raising the issue and helping me through the development of the fixes.

Kieran Jacobsen