If you are facing difficulty while systematically reading log files. Then you are in the right place.
Simple doing cat to read log file is a kind basic way to inspect log file. But when you want to filter data in an organized way based on time, then we use awk or grep.
The awk is a popular command-line tool used to manipulate data in files and generate a report based on a given pattern in the Linux System. While grep is simple pattern tool that search pattern using a regular expression.
In this article, we will filter /var/log/syslog file based on date range using both awk and grep command-line tools in the Linux System.
Date and Time range extraction of a log file
First, we filter the syslog file based on the time range using awk.
Syntax
awk -v start=00:00 -v stop=09:00 'start <= $2 && $2 < stop' file.log
$ sudo awk -v start=07:00 -v stop=19:00 'start <= $2 && $2 < stop' /var/log/syslog
Output
Dec 11 19:39:19 LINUX dbus-daemon[2742]: [session uid=1000 pid=2742] Activating via systemd: service name='org.freedesktop.Tracker1' unit='tracker-store.service' requested by ':1.3' (uid=1000 pid=2740 comm="/usr/libexec/tracker-miner-fs " label="unconfined")
Dec 11 19:39:19 LINUX systemd[2727]: Starting Tracker metadata database store and lookup manager...
Dec 11 19:39:19 LINUX systemd[1]: session-23.scope: Succeeded.
Dec 11 19:39:19 LINUX dbus-daemon[2742]: [session uid=1000 pid=2742] Successfully activated service 'org.freedesktop.Tracker1'
Replace start=HH: MM with starting hour and minute, stop=HH: MM with final hour and minute.
If you want to check the log more easily and simple way, then you can follow the grep command utility.
I’ll show a few examples using grep like I want to check the log for Today, then I’ll pass below command.
Syntax
$ sudo grep -E "pattern" logfile
$ sudo grep -E " Dec 11" /var/log/syslog
Output
Dec 11 12:07:35 LINUX /usr/libexec/gdm-x-session[1231]: (==) Log file: "/var/lib/gdm3/.local/share/xorg/Xorg.0.log", Time: Fri Dec 11 12:07:35 2020
Dec 11 12:08:39 LINUX /usr/libexec/gdm-x-session[2812]: (==) Log file: "/home/shen/.local/share/xorg/Xorg.1.log", Time: Fri Dec 11 12:08:39 2020
Dec 11 12:09:04 LINUX gnome-shell[3019]: GNOME Shell started at Fri Dec 11 2020 12:08:55 GMT+0530 (IST)
Dec 11 14:24:41 LINUX /usr/libexec/gdm-x-session[1208]: (==) Log file: "/var/lib/gdm3/.local/share/xorg/Xorg.0.log", Time: Fri Dec 11 14:24:41 2020
Dec 11 14:25:13 LINUX /usr/libexec/gdm-x-session[2783]: (==) Log file: "/home/shen/.local/share/xorg/Xorg.1.log", Time: Fri Dec 11 14:25:13 2020
Dec 11 14:25:40 LINUX gnome-shell[3049]: GNOME Shell started at Fri Dec 11 2020 14:25:31 GMT+0530 (IST)
Now we will explain to you how this command work “grep”. A grep is used to search regular patterns in the provided file.
The Parameters -E is called extended -regexp that is used to set pattern and last you need to provide log file or log directory.
To get data between a range of date examples from 01 Dec to 10 Dec, use the below command.
Syntax
$ sudo grep -E "pattern1 | pattern 2" logfile
$ sudo grep -E "Dec 1|Dec 10" syslog
Output
Dec 11 19:48:04 LINUX systemd[1]: Starting Hostname Service...
Dec 11 19:48:04 LINUX dbus-daemon[856]: [system] Successfully activated service 'org.freedesktop.hostname1'
Dec 11 19:48:04 LINUX systemd[1]: Started Hostname Service.
Dec 11 19:48:34 LINUX systemd[1]: systemd-hostnamed.service: Succeeded.
Dec 11 19:50:01 LINUX CRON[39178]: (shen) CMD (~/duckdns/duck.sh >/dev/null 2>&1)
Dec 11 19:51:58 LINUX dbus-daemon[856]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.320' (uid=1000 pid=38994 comm="/usr/bin/gnome-screenshot --gapplication-service " label="unconfined")
Dec 11 19:51:58 LINUX systemd[1]: Starting Hostname Service...
Dec 11 19:51:58 LINUX dbus-daemon[856]: [system] Successfully activated service 'org.freedesktop.hostname1'
Dec 11 19:51:58 LINUX systemd[1]: Started Hostname Service.
Now we will show you How to filter data using sed command line.
Syntax
$ sudo sed -n '/pattern1/,/pattern2/p' logfile
To get data between a range of dates with certain time examples from 11 Dec 18:45:06 to 11 Dec 19:00:01, use the below command.
$ sudo sed -n '/Dec 11 18:45:06 /,/Dec 11 19:00:01/p' /var/log/syslog
Output
Dec 11 18:45:06 LINUX dbus-daemon[856]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.301' (uid=1000 pid=35830 comm="/usr/bin/gnome-screenshot --gapplication-service " label="unconfined")
Dec 11 18:45:06 LINUX systemd[1]: Starting Hostname Service...
Dec 11 18:45:06 LINUX dbus-daemon[856]: [system] Successfully activated service 'org.freedesktop.hostname1'
Dec 11 18:45:06 LINUX systemd[1]: Started Hostname Service.
Dec 11 18:45:36 LINUX systemd[1]: systemd-hostnamed.service: Succeeded.
Dec 11 18:47:22 LINUX dbus-daemon[856]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.303' (uid=1000 pid=35830 comm="/usr/bin/gnome-screenshot --gapplication-service " label="unconfined")
Dec 11 18:47:22 LINUX systemd[1]: Starting Hostname Service...
Dec 11 18:47:22 LINUX dbus-daemon[856]: [system] Successfully activated service 'org.freedesktop.hostname1'
Dec 11 18:47:22 LINUX systemd[1]: Started Hostname Service.
Dec 11 18:47:52 LINUX systemd[1]: systemd-hostnamed.service: Succeeded.
Dec 11 18:50:01 LINUX CRON[36057]: (shen) CMD (~/duckdns/duck.sh >/dev/null 2>&1)
Dec 11 18:55:01 LINUX CRON[36206]: (shen) CMD (~/duckdns/duck.sh >/dev/null 2>&1)
Dec 11 19:00:01 LINUX CRON[36473]: (shen) CMD (~/duckdns/duck.sh >/dev/null 2>&1)
We will explain how this command work “sed” is a stream editor used to search, remove, replace, insert, and other functions.
The -n parameter with sed will not print the output of each line. Now the /pattern1/,/pattern2/ will be the expression
In the last -p parameters suggest sed verify the next expression according to a given pattern
After all this you need to provide the log file or log file location.
If you have any query and issue related to the Date filter. Feel free to ask your query in the comment section
Innovative tech mind with 12 years of experience working as a computer programmer, web developer, and security researcher. Capable of working with a variety of technology and software solutions, and managing databases.
