Creating Practical Examples of SIEM for Linux Systems?

LearningTechAndDev

New Member
Joined
Sep 25, 2023
Messages
11
Reaction score
0
Credits
197
Hello Everyone
I am mostly a web developer and most of my applications are hosted on linux servers. I am looking to improve my knowledge of two things:

1) Improve my knowledge of overall system security

2) Improve my knowledge of a platform called Elasticsearch

With regards to Elasticsearch, I've been self-studying it for the past year and I find it extremely enjoyable and entertaining. Along my journey, I realized Elasticsearch has something called the Beats Library, which are data extraction agents that you place on target servers which then ship all kinds of data back to your elasticsearch platform. For example, if your target servers are a bunch of Apache servers running on Ubuntu, RedHat, etc..., you can install a Metricbeat on the linux-apache server to capture system resource information, Filebeat to capture apache logs, access logs, etc... And within elasticsearch, you can configure various alerts to notify you of activities you want to be immediately notified of (excessive http 403 requests from specific IP addresses, too many failed SSH access attempts, unauthorized attempts to update module librarys on server, disk usage above a particular level etc...). And since all the logs are consolidated in one place in elasticsearch, you can easily query all the data coming from the fleets of servers you deployed out in the field.

I recently came across the term SIEM, which to me seems like it describes exactly what I stated in the paragraph above. IF that is correct, does that mean Elasticsearch can be a strong consideration as a robust SIEM tool?

And if it is correct, I want to produce a some practical demonstrations of using Elasticsearch, Beats and anything else as an SIEM tool. But given I'm not a senior SIEM specialist, I don't know what would be considered practical demonstrations. Can anyone list some things you think would be practical demonstrations of elasticsearch as a SIEM tool?


My goal is to host 1 hour to 2 hour webinar to do these practical demonstrations. The webinar will be free, recorded and shared via YouTube for those who missed it. The audience will be to other full stack developers such as myself who have basic day to day linux skills from a programmer's perspective (as opposed to a hardcore Linux systems admin). I want to show people how quiclky and easily they can set up Elasticsearch for SIEM purposes.

As of today, I already produced 40 videos explaining the basic building blocks of elasticsearch on my channel here: https://www.youtube.com/@evermighttech/videos . But they are just "building blocks". I know want to move on to full-fledged practical demonstrations and sharing complete code explanations.

So to summarize my questions:
1) Does it sound like Elasticsearch is a reasonable SIEM Tool? If yes, move to quesiton 2 below:
2) What are some practical demonstrations you would like to see from Elasticsearch as an SIEM tool.
 


Hmm why not just implement splunk and run some reports. Analyzing and reporting security threats is really a bit more complicated than that. Threats also evolve just creating some basic rules and launching some notifications probably not all that affective. If you are just doing system health you could also do something like snmp notifications. There tare lots of home brew things you can do but I would leave SIEM to folks that specialize in it.
 
Hmm why not just implement splunk and run some reports. Analyzing and reporting security threats is really a bit more complicated than that. Threats also evolve just creating some basic rules and launching some notifications probably not all that affective. If you are just doing system health you could also do something like snmp notifications. There tare lots of home brew things you can do but I would leave SIEM to folks that specialize in it.
I just ended up creating the basics of an observability and SIEM platform using Elasticsearch, Kibana, Logstash and Beats and I share my entire code base here:


It only takes about 5 to 10 minutes to spin up, but I provided a 1 hour tutorial that explains a few more things:

I plan to create a series of 10 youtube episodes that explains the code line by line, so that others can improve upon what I did or repurpose it for other applications.

The challenge: i have to support various businesses, and they all have several linux servers. Each linux server has more than one wordpress website installed on them. It is too time consuming for me to login to each individual wordpress website and each individual linux server to review out of date plugins, new wordpress plugin integrations, bruteforce ssh login attempts, unusual system resource usage, unusual mysql performance, unusual apache performance etc...

My solution: use the beats library to ship customer server operational data to elasticsearch. Built a wordpress plugin that scans the health of the wordpress website and ship information to elasticsearch. Build bash scripts that ship logs from fail2ban, clam anti virus etc... to elasticsearch. Use Kibana to visualize all the data in elasticsearch. Use logstash to relay any important notification from elasticsearch and kibana to my email.

Hopefully this is a good starting point?
 
And some examples of dashboards that are configured with the Elastic/Kibana stack

filebeat-apache.png


wp-siem-metricbeat-host.png
siem-intro-08.jpg
siem-intro-05.jpg
 


Top