Installation & Configuration of ClamAV Antivirus on Ubuntu 18.04
(with On-Access Scanning)
But I thought Linux (and Macs) aren’t affected by Viruses?
Often people ask is Antivirus necessary on Linux systems? The answer is it really depends.
Some may need it for compliance purposes, for example some QSAs (PCI DSS) would consider Linux these days to be “commonly affected by malicious software”. Recently I found myself in this exact situation, hence my reason for writing this article.
Other situations might include a Linux server that serves up file-based content to Windows systems, such as a file server, or a mail server that processes email attachments.
In any event, for anyone who has looked into Antivirus solutions for Linux, the options out there are pretty slim. What you will find though is a lot of references to a software named “ClamAV”.
What is covered in this article?
In this article, I’ll be sharing and documenting what I’ve learned during a recent situation where I had to install, setup, and configure ClamAV to run on Ubuntu 18.04 systems using real-time or as ClamAV calls it “on-access scanning”.
As I discovered recently, ClamAV is not perfect, and still leaves a lot to be desired, especially around their “on-access scanning” feature.
What is ClamAV Antivirus?
Beyond ClamAV obviously being an Antivirus solution for Linux, what exactly is ClamAV and how did it get to where it is today?
ClamAV is an open-source Antivirus solution that has been around for quite a while and is most commonly used in an integrated fashion with mail servers for email scanning. With recent features like On-Access Scanning, it appears to be trying to expand into a more general-use Antivirus solution.
In 2007, ClamAV was acquired by a company named Sourcefire, who later in 2013 was acquired by Cisco and is now maintained by a division of Cisco named Cisco Talos.
Prerequisites for following this tutorial
We will be working with the following OS and ClamAV version, if you are using a different Linux variant or ClamAV version, your mileage may vary.
- Ubuntu 18.04.4 LTS
- ClamAV (Ubuntu Repo Package: 0.102.3+dfsg-1, latest available at the time)
- AWS EC2 Instance (t3.medium, 2 cores, 4GB memory)
Components of ClamAV
ClamAV is made up of a number of a components which allow it to be used in a number of different ways depending on your use case.
We will be using the following components (or daemons):
- clamd (or clamav-daemon) — Daemon that loads the virus database definitions into memory, and handles scanning of files when instructed to do so by clients such as clamdscan or clamonacc.
- freshclam daemon (or clamav-freshclam) — Daemon that periodically checks for virus database definition updates, downloads, installs them, and notifies clamd to refresh it’s in-memory virus database cache.
- clamdscan — Utility that allows you to scan the filesystem and ask clamd to scan a given set files.
- clamonacc daemon — Similar to clamdscan, but listens for file operations and asks clamd to scan files with activity. This daemon provides the On-Access Scanning functionality.
Memory Requirements Disclaimer
When the “clamd” daemon service is running, in my testing it utilizes around 1GB of memory perpetually regardless of whether it is actively scanning anything. This is due to it loading the virus definitions database into memory, this reduces the latency and overhead of scanning individual files substantially.
If your use case revolves around scanning a given directory on a regular set schedule, or at an infrequent event driven and latency irrelevant frequency then you’d be better suited to use the “clamscan” utility which loads the virus definitions database into memory at the time of scan, and releases the memory at the end of the scan. This is not covered in this article.
Installation & Configuration of ClamAV (without On-Access Scanning)
The following steps will instruct you how to install and configure ClamAV for use as a low-latency scanning solution for ad-hoc or scheduled file scans using the clamdscan utility. If you want real-time or “on-access” scanning, follow the instructions in this section, and then follow the additional steps in the “Setup & Configuring ClamAV On-Access Scanning” section.
1. Install the “clamav-daemon” Package
sudo apt-get install clamav-daemon
After installation of the above package, you’ll have the following systemd services installed:
- clamav-freshclam.service (enabled & running)
- clamav-daemon.service (disabled)
When the “clamav-freshclam” service started up, it automatically downloaded the latest virus database definitions to /var/lib/clamav
.
2. Enable & start the “clamav-daemon” service
sudo systemctl enable clamav-daemon
sudo systemctl start clamav-daemon
When you issue the systemctl start clamav-daemon
command it completes immediately, but in-fact the service is still loading virus database definitions into memory, and until that is done, the socket that is needed for “clamdscan” to talk to “clamd” won’t be available. To confirm “clamd” is fully started you can check it’s log file or the socket.
Log file
ubuntu@ip-172-31-81-67:~$ tail /var/log/clamav/clamav.log
Sun Jun 28 19:08:32 2020 -> Portable Executable support enabled.
Sun Jun 28 19:08:32 2020 -> ELF support enabled.
Sun Jun 28 19:08:32 2020 -> Mail files support enabled.
Sun Jun 28 19:08:32 2020 -> OLE2 support enabled.
Sun Jun 28 19:08:32 2020 -> PDF support enabled.
Sun Jun 28 19:08:32 2020 -> SWF support enabled.
Sun Jun 28 19:08:32 2020 -> HTML support enabled.
Sun Jun 28 19:08:32 2020 -> XMLDOCS support enabled.
Sun Jun 28 19:08:32 2020 -> HWP3 support enabled.
Sun Jun 28 19:08:32 2020 -> Self checking every 3600 seconds.
Socket
ubuntu@ip-172-31-81-67:~$ ls -l /var/run/clamav/clamd.ctl
srw-rw-rw- 1 clamav clamav 0 Jun 28 19:08 /var/run/clamav/clamd.ctl
If the above command fails with: cannot access '/var/run/clamav/clamd.ctl': No such file or directory
the “clamd” service hasn’t finished starting.
3. Testing scan functionality
Try scanning your home directory using the “clamd” client/utility “clamdscan”:
cd ~
sudo clamdscan
You’ll likely get an error like so:
ubuntu@ip-172-31-81-67:~$ sudo clamdscan
/home/ubuntu/.lesshst: Access denied. ERROR
/home/ubuntu/.viminfo: Access denied. ERROR
/home/ubuntu/.ssh: lstat() failed: Permission denied. ERROR
/home/ubuntu/.gnupg: lstat() failed: Permission denied. ERROR
/home/ubuntu/.cache: lstat() failed: Permission denied. ERROR----------- SCAN SUMMARY -----------
Infected files: 0
Total errors: 5
Time: 0.017 sec (0 m 0 s)
This is due to the fact that by default on Ubuntu “clamd” drops privileges and runs as the user “clamav”. By running clamdscan as we did above, we were asking “clamd” to use the permissions that the “clamav” user has to scan our home directory.
Instead, the “clamdscan” utility has a parameter we can pass to it when invoking that allows it to pass a file descriptor to clamd giving clamd the necessary permissions to scan the file(s).
Try again:
sudo clamdscan --fdpass
It works this time:
ubuntu@ip-172-31-81-67:~$ sudo clamdscan --fdpass
/home/ubuntu: OK----------- SCAN SUMMARY -----------
Infected files: 0
Time: 0.015 sec (0 m 0 s)
One final test, let’s download a virus test file and run the scan again.
wget www.eicar.org/download/eicar.com
sudo clamdscan --fdpass
You should see the eicar file detected:
ubuntu@ip-172-31-81-67:~$ sudo clamdscan --fdpass
/home/ubuntu/eicar.com: Win.Test.EICAR_HDB-1 FOUND----------- SCAN SUMMARY -----------
Infected files: 1
Time: 0.005 sec (0 m 0 s)
4. Scheduling periodic scans with “clamdscan” and “cron” daemon
Now that we have ClamAV installed and running, we may we want to setup periodic scanning of our system for malware. We can do so using Ubuntu’s built-in cron daemon.
Assuming we want to scan our system weekly, let’s create a new crontab file @ /etc/cron.d/clamdscan
.
Warning: Any files detected as malware when the cron runs clamdscan will be moved, including potential false positives — so if you’re running on a production system, do so at your own risk.
sudo mkdir /root/quarantine
echo "0 1 * * 0 root /usr/bin/clamdscan --fdpass --log=/var/log/clamav/clamdscan.log --move=/root/quarantine /" | tee /etc/cron.d/clamdscan
Unraveling the cron entry
As we can see from above 0 1 * * 0
we are scheduling this scan to run every Sunday at 1 AM.
We are running clamdscan
as the root
user, this is important if we are planning to scan the entire system, as well as if we want to move infected files somewhere that only root has access to (see below)
We are also using the --fdpass
functionality we previously discussed.
What’s new is the --log
, --move
and /
properties:
--log=/var/log/clamav/clamdscan.log
tells “clamdscan” to log scan results to /var/log/clamav/clamdscan.log
.
--move=/root/quarantine
tells “clamdscan” to move any detected malware to the /root/quarantine
directory.
/
tells “clamdscan” we want to scan the entire root filesystem.
5. Testing full system scan
We can test this scan by running the command manually:
Warning: Any files detected as malware will be moved when you run this command, including potential false positives — so if you’re running on a production system, do so at your own risk.
sudo /usr/bin/clamdscan --fdpass --log=/var/log/clamav/clamdscan.log --move=/root/quarantine /
On a fresh Linux system it shouldn’t take too long to run. While it’s running you’ll likely see a number of WARNING:
and ERROR
messages appear, this is to be expected — ClamAV will try to scan everything and anything even if it’s potentially obvious it wouldn’t be able to. When it’s finished scanning you’ll see a summary similar to this:
----------- SCAN SUMMARY -----------
Infected files: 2
Total errors: 44083
Time: 40.186 sec (0 m 40 s)
We see it detected two infected files, and 44083 errors! Let’s first see what files it detected as infected:
ubuntu@ip-172-31-81-67:~$ sudo grep FOUND /var/log/clamav/clamdscan.log
/home/ubuntu/eicar.com: Win.Test.EICAR_HDB-1 FOUND
/root/quarantine/eicar.com: Win.Test.EICAR_HDB-1 FOUND
Interesting, so it found our eicar.com file that we downloaded previously. It also found the same file in /root/quarantine
. Remember, how we told “clamdscan” to move all infected files to /root/quarantine? Well it did, and then later on in the scan it scanned the /root/quarantine directory and found the file again. This seems redundant doesn’t it? We can exclude this directory from scanning, we’ll do so shortly.
What about those 44083 errors?
Cleaning up the log file a bit, we can see that the errors seem to fall into two categories:
ubuntu@ip-172-31-81-67:~$ sudo grep ERROR /var/log/clamav/clamdscan.log | cut -d":" -f2 | grep "^ " | sort | uniq -c | sort -k1 -n
918 Access denied. ERROR
7669 Can't read file ERRORubuntu@ip-172-31-81-67:~$ sudo grep WARNING /var/log/clamav/clamdscan.log | cut -d":" -f3 | grep "^ " | sort | uniq -c | sort -k1 -n
260 Not supported file type
Digging deeper, we can see that the errors originate in the following parent directories:
ubuntu@ip-172-31-81-67:~$ sudo grep ERROR /var/log/clamav/clamdscan.log | cut -d"/" -f2 | cut -d"/" -f1 | sort | uniq -c | sort -k1 -n
10 proc
402 var
43410 sysubuntu@ip-172-31-81-67:~$ sudo grep "WARNING:" /var/log/clamav/clamdscan.log | cut -d"/" -f2 | cut -d"/" -f1 | sort | uniq -c | sort -k1 -n
1 sys
1 var
33 run
79 snap
147 dev
Analyzing those results, we see that /proc
, /sys
, /run
, /dev
, and /snap
, are special directories on Linux containing process, device, socket files and snap package mounts. We can likely exclude those from the scan.
Trickier is the /var
directory, if we dig deeper:
/var
ubuntu@ip-172-31-81-67:~$ sudo grep "^/var/" /var/log/clamav/clamdscan.log | cut -d"/" -f3 | cut -d"/" -f2 | sort | uniq -c | sort -k1 -n
402 lib
ubuntu@ip-172-31-81-67:~$ sudo grep "^/var/" /var/log/clamav/clamdscan.log | cut -d"/" -f4 | cut -d"/" -f3 | sort | uniq -c | sort -k1 -n
402 lxcfs
ubuntu@ip-172-31-81-67:~$ sudo grep "^/var/" /var/log/clamav/clamdscan.log | cut -d"/" -f5 | cut -d"/" -f4 | sort | uniq -c | sort -k1 -n
402 cgroup
ubuntu@ip-172-31-81-67:~$ sudo grep "^/var/" /var/log/clamav/clamdscan.log | cut -d"/" -f6 | cut -d"/" -f5 | sort | uniq -c | sort -k1 -n
67 blkio
134 devices
201 memory
To reduce the noise we can also likely exclude /var/lib/lxcfs/cgroup
from the scan.
In your situation depending on how your server(s) are configured and what applications are installed and running on them, you may need to do some additional investigation as we’ve done above for other directories to reduce additional noise.
We now have a list of directories to exclude from the weekly scans:
- /root/quarantine
- /proc
- /sys
- /run
- /dev
- /snap
- /var/lib/lxcfs/cgroup
How do we go about excluding them?
6. Excluding directories from scanning using “clamd.conf”
There is a configuration file located at /etc/clamav/clamd.conf
, which allows a finer grain of detail configuring how the clamd daemon, as well as the clamdscan utility function.
To exclude directories from being scanned we can use the ExcludePath
configuration property, let’s add the directories we want to exclude to the “clamd.conf” file.
printf "ExcludePath ^/proc\nExcludePath ^/sys\nExcludePath ^/run\nExcludePath ^/dev\nExcludePath ^/snap\nExcludePath ^/var/lib/lxcfs/cgroup\nExcludePath ^/root/quarantine\n" | sudo tee -a /etc/clamav/clamd.conf
To apply the configuration restart the “clamav-daemon” service.
sudo systemctl restart clamav-daemon
7. Testing full system scan (again)
sudo /usr/bin/clamdscan --fdpass --log=/var/log/clamav/clamdscan.log --move=/root/quarantine /
The results should look much cleaner now:
ubuntu@ip-172-31-81-67:~$ sudo /usr/bin/clamdscan --fdpass --log=/var/log/clamav/clamdscan.log --move=/root/quarantine /
--------------------------------------
/snap: Excluded
/run: Excluded
/dev: Excluded
/proc: Excluded
/sys: Excluded
/root/quarantine: Excluded
/var/lib/lxcfs/cgroup: Excluded
WARNING: /var/lib/lxd/unix.socket: Not supported file type----------- SCAN SUMMARY -----------
Infected files: 0
Total errors: 1
Time: 235.497 sec (3 m 55 s)
You should now have a weekly scan setup that writes it’s results to /var/log/clamav/clamdscan.log
.
For many people, what we’ve just covered is enough for their purposes. If however, you need or want ClamAV’s On-Access Scanning functionality feel free to continue with the following sections.
Caveat of ClamAV’s On-Access Scanning
In the last section we went over how to install, configure and test the “clamd” daemon and it’s associated client utility “clamdscan” — along with scheduling a weekly full system scan with appropriate exclusion rules.
Before we get into setting up and configuring ClamAV’s On-Access Scanning, I want to cover how On-Access Scanning works and some of the caveats of using it as of version 0.102.3.
How does On-Access Scanning work?
The official On-Access Scanning documentation describes this better than I could ever aim to:
ClamAV’s On-Access Scanning system uses a scheme called Dynamic Directory Determination (DDD for short) which is a shorthand way of saying that it tracks the layout of every directory specified with
OnAccessIncludePath
dynamically, and recursively, in real time. It does this by leveraging inotify……it leverages a kernel api called fanotify to block processes from attempting to access malicious files.
Caveats
ClamAV’s On-Access Scanning functionality in it’s current state, has quite a few caveats some of which are not documented. Below you’ll find the ones that I’ve ran head first into, and others that you should be aware of if you intend to use the feature.
1. Inotify watchpoint limitations
Given On-Access Scanning utilizes inotify, there are limitations to the number of directories that a given process (clamonacc is no exception) can monitor at any given time. By default this value is 8192
, however it can be increased, for example (to 524288):
echo 524288 | sudo tee -a /proc/sys/fs/inotify/max_user_watches
To persist the above change append it to the sysctl.conf file:
echo "fs.inotify.max_user_watches = 524288" | sudo tee -a /etc/sysctl.conf
2. fdpass functionality is enabled by default (DON’T USE --fdpass)
On-Access Scanning functionality is provided by a daemon launched using the “clamonacc” command. A number of similarities exist between “clamdscan” and “clamonacc”, given they are both “clamd” clients. This includes the --fdpass
property and it’s associated functionality.
Previously, I ran into issues using clamonacc while attempting to enable the file descriptor passing functionality. It just wouldn’t work, I kept running into the following error message in the clamonacc logs whenever file system activity would occur:
ERROR: ClamCom: TIMEOUT while waiting on socket (recv)
At the time, I found others having the same issue, and assumed it wasn’t a “problem between the keyboard and the chair”, and therefore proceeded to open a bug report with the ClamAV team: https://bugzilla.clamav.net/show_bug.cgi?id=12563
Later, I realized after reading this ClamAV blog post from last year:
Generally, if you are running clamd on the same system as clamonacc, you will be using a local unix socket and file descriptor passing is enabled by default.
Sure enough, if you run “clamonacc” without the --fdpass
property it instructs “clamd” to scan files in the same way that “clamdscan” does when provided with the --fdpass
property.
3. Watching directory paths that contain special files
In the official On-Access Scanning documentation they advise against attempting to monitor the entire system, and in-fact even prevent you from even setting OnAccessIncludePath /
.
As can be seen from the following open bug ticket, and forum post, if you try to have “clamonacc” watch a directory hierarchy that contains special files such as sockets or other mount points, it will fail to start up altogether.
For instance if we tried to watch /var (excluding /var/log), we’d see the following error in clamonacc’s log:
--------------------------------------
ClamInotif: watching '/var' (and all sub-directories)
ClamInotif: excluding '/var/log' (and all sub-directories)
ERROR: ClamInotif: could not watch path '/var', 3
We can see from the following command, that the /var folder contains one or more socket
files, which is tripping up clamonacc.
ubuntu@ip-172-31-81-67:~$ sudo find /var -exec stat -c%F {} \; | sort | uniq
directory
regular empty file
regular file
socket
symbolic link
Specifically:
ubuntu@ip-172-31-81-67:~$ sudo find /var -print -exec stat -c%F {} \; | grep "^socket" -B1
/var/lib/lxd/unix.socket
socket
Other types of files that I’ve been tripped up by include:
- character special file
- fifo
One way to work-around this issue, is to be more granular in the directories you’re deciding to watch. We could exclude /var/lib/lxd
by choosing to include each of these directories directly (with exception to /var/lib/lxd of course):
ubuntu@ip-172-31-81-67:~$ ls /var/lib
AccountsService landscape python
amazon logrotate snapd
apt lxcfs sudo
clamav lxd systemd
cloud man-db ubuntu-release-upgrader
command-not-found misc ucf
dbus mlocate unattended-upgrades
dhcp os-prober update-manager
dpkg pam update-notifier
git plymouth ureadahead
grub polkit-1 usbutils
initramfs-tools private vim
ubuntu@ip-172-31-81-67:~$ ls /var/
backups crash local log opt snap tmp
cache lib lock mail run spool
That is totally unwieldy, and what happens when new directories are created in /var/lib
or /var
…
There isn’t a good solution to this at this time, so be mindful of this caveat if you’re planning to watch large sections of your filesystem and/or sections that may end up containing special files.
4. Inode exhaustion with on-access scanning /tmp
Typically it would make sense to setup on-access scanning for /tmp, given many web applications use /tmp by default as a temporary space when receiving (questionable?) file uploads from end-users.
That said, you need to be careful doing so as I discovered after some production systems I was working on ran into a relatively tough issue to diagnose. On systems with a dedicated “/tmp” partition or tmpfs configuration, the /tmp partition became unwritable due to inode exhaustion, more critically on systems without a disparate partition “/” became unwritable. What made it really difficult to diagnose was the fact that “find”, “ls”, etc… did not show any files in /tmp. Likewise, “lsof” didn’t show deleted files being held open by any applications.
ClamAV’s components (“clamd”, “clamonacc”, etc…) use /tmp as well as a scratch space while it carries out scans, and this can cause a situation where clamonacc runs into a fatal error as it appears it is attempting to scan clamd’s or it’s own tmp files (race condition?). It appears, normally, this wouldn’t be an issue because the properties “OnAccessExcludeUname” and “OnAccessExcludeRootUID” would skip scanning these files, however, if a file is so short lived that “clamonacc” cannot determine who is the author of the file action, it will scan it anyways. When clamonacc crashes (but does not exit) as seen in the error message below after scanning a variable number of it’s own or “clamd’s” tmp files, it ends up subsequently holding an inode for every subsequent file created in /tmp, until the inodes are exhausted or the “clamonacc” process/daemon is killed. The following bug report goes into more detail: https://bugzilla.clamav.net/show_bug.cgi?id=12587 (as of writing, the bug report is still hidden from public view):
ClamInotif: DELETE - removing /tmp/clamav-2af11edb31faecf2304c166ac46a0e27.tmp/rfc2397 from /tmp/clamav-2af11edb31faecf2304c166ac46a0e27.tmp with wd:921
Clamonacc: onas_clamonacc_exit(), signal 11
ERROR: Clamonacc: clamonacc has experienced a fatal error, if you continue to see this error, please run clamonacc with --debug and report the issue and crash report to the developpers
Clamonacc: attempting to stop event consumer thread ...
A possible workaround is to adjust clamd’s config file and set TemporaryDirectory to a different directory that “clamonacc” isn’t monitoring, for instance /var/clamav/tmp
:
sudo mkdir -p /var/clamav/tmp
sudo chown clamav:root /var/clamav/tmp
sudo chmod 770 /var/clamav/tmp
printf "TemporaryDirectory /var/clamav/tmp" | sudo tee -a /etc/clamav/clamd.conf
5. AppArmor usr.sbin.clamd profile restrictions
By default, AppArmor is installed and enabled on Ubuntu 18.04, and it heavily restricts what the “clamd” daemon can do. If you have “clamonacc” watch a directory that is not explicitly allowed in the AppArmor profile (for example /bin), you’ll see something like the following in your clamonacc log.
/bin/grep: Can't open file or directory ERROR
ClamMisc: internal issue (client failed to scan)
/bin/ps: Can't open file or directory ERROR
ClamMisc: internal issue (client failed to scan)
/bin/grep: Can't open file or directory ERROR
ClamMisc: internal issue (client failed to scan)
/bin/ps: Can't open file or directory ERROR
ClamMisc: internal issue (client failed to scan)
To disable AppArmor from enforcing restrictions on “clamd” you can set the profile to complain mode.
Warning: Doing so reduces some of the safety mechanisms that AppArmor has put in place to harden clamd, do so at your own risk.
sudo aa-complain /usr/sbin/clamd
If the above command fails with Command 'aa-complain' not found
you may need to install the apparmor-utils package:
sudo apt-get install apparmor-utils
6. OnAccessPrevention (optional) performance implications
Another caveat is the performance hit when using OnAccessPrevention, a feature which uses the fanotify kernel API to block access to malicious files.
How it works
Flags fanotify to block any triggered events on monitored files, which allows ClamAV to scan affected files to determine if those events should be allowed to proceed.
https://blog.clamav.net/2016/03/configuring-on-access-scanning-in-clamav.html
Implications
Typical file access operations can be quite fast depending on your underlying storage system, for example if were to create a test file that’s roughly 169KB:
for i in {1..8788}; do echo " test file contents " >> test_file.txt; done
Let’s clear any file system caches and test how long it takes to cat our test file:
ubuntu@ip-172-31-81-67:~$ for i in {1,2,3}; do sudo sync; echo $i | sudo tee -a /proc/sys/vm/drop_caches; done
1
2
3
ubuntu@ip-172-31-81-67:~$ time cat test_file.txt &>/dev/null # No Cachereal 0m0.010s
user 0m0.002s
sys 0m0.000s
ubuntu@ip-172-31-81-67:~$ time cat test_file.txt &>/dev/null # Cachereal 0m0.002s
user 0m0.001s
sys 0m0.001s
Let’s turn on OnAccessPrevention with clamonacc monitoring the same directory that test_file.txt
resides in as well as the bin directories where cat
and time
binaries are located to see how the I/O time changes:
ubuntu@ip-172-31-81-67:~$ for i in {1,2,3}; do sudo sync; echo $i | sudo tee -a /proc/sys/vm/drop_caches; done
1
2
3
ubuntu@ip-172-31-81-67:~$ time cat test_file.txt &>/dev/null # No Cachereal 0m0.023s
user 0m0.003s
sys 0m0.000s
ubuntu@ip-172-31-81-67:~$ time cat test_file.txt &>/dev/null # Cachereal 0m0.031s
user 0m0.002s
sys 0m0.000s
From the above tests, we can see that I/O access times increased by at least double in non-cached tests, and in cached tests it’s 30x slower! This level of slow down you can actually feel when running commands on the command line.
Depending on what your server is doing, this level of I/O performance impact could be crippling.
I would advise against using OnAccessPrevention unless you are monitoring very specific directories, and know that low latency I/O performance is not critical.
7. Lack of filtering options based on access type (read vs. write vs. execute)
Lastly, and likely ClamAV’s On-Access Scanning’s most limiting quality is it’s inability to filter based on certain types of access operations.
When using On-Access Scanning to watch a directory for file system activity, there is no ability to configure it to only watch for say file executions or writes.
The implications of this are that if you watch certain directories that have large amounts of read activity, such as system or library directories (/bin, /usr/bin, or /usr/lib, etc…). ClamAV can end up being flooded with scan requests, and in some cases actually become overwhelmed and bring a system to a crawl.
In reality if you try to setup ClamAV On-Access Scanning to monitor your entire system in real-time, you’ll quickly find yourself:
- Re-evaluating I/O activity patterns to feed additional directory exclusion rules
- Monitoring for special files (as described previously) that may break clamonacc’s ability to start up in the future
- Monitoring resource utilization
- Tweaking / restricting clamd performance.
As it currently stands, ClamAV’s On-Access Scanning feature is far from being able to operate in a hands-off manner when used to monitor an entire system and/or a system with lots of read activity needing to be watched.
Setup & Configuring ClamAV On-Access Scanning
With the above caveats in-mind, let’s take our existing ClamAV installation and set it up to handle On-Access Scanning.
Our goals for the real-time scanning functionality will be:
- Monitor directories that are entry points for new files contributed by users of the system (specifically
/home
and/var/www
) - Move any files identified as being infected to
/root/quarantine
- Scan in parallel to the file access operation (do not block access with
OnAccessPrevention
)
1. Update clamd.conf with OnAccess* properties
First thing we’ll want to do is update our clamd.conf file with some properties that “clamonacc” will read to inform it’s behaviour.
printf "OnAccessIncludePath /home\nOnAccessIncludePath /var/www\nOnAccessExcludeUname clamav\nOnAccessExcludeRootUID true" | sudo tee -a /etc/clamav/clamd.conf
Unraveling that we can see that we are using OnAccessIncludePath
to tell “clamonacc” to watch for changes to files recursively in /home
and /var/www
.
We are also excluding file operations carried out by users clamav
and root
using OnAccessExcludeUname
and OnAccessExcludeRootUID
. This is important as we don’t want to end up in a potential endless loop situation where “clamonacc” running as root
or “clamd” running as clamav
reads a file to scan it, and ends up triggering another scan as part of that operation.
Note: If you choose to monitor /tmp, be sure to consider the implications mentioned prior in the caveat “4. Inode exhaustion with on-access scanning /tmp”.
2. Create a systemd unit file for “clamonacc”
In Ubuntu there unfortunately isn’t a provided systemd unit file for “clamonacc”. Thankfully someone that goes by the Github name ChadDevOps has created one for us.
# /etc/systemd/system/clamonacc.service
[Unit]
Description=ClamAV On Access Scanner
Requires=clamav-daemon.service
After=clamav-daemon.service syslog.target network.target
[Service]
Type=simple
User=root
ExecStart=/usr/bin/clamonacc -F --log=/var/log/clamav/clamonacc --move=/root/quarantine
Restart=on-failure
RestartSec=120s
[Install]
WantedBy=multi-user.target
An issue I mentioned prior which I ran into as it seems others have, is that “clamd” tells systemd as part of the “clamav-daemon” service that it has finished starting up well before it’s finished loading the virus definition database into memory and created it’s socket file needed for “clamonacc” to talk to it.
Our Github unit file appears to solve this issue by setting the following properties under the [Service]
section:
Restart=on-failure
RestartSec=120s
However, this will still result in the following error messages showing up in our “clamonacc” log file:
ERROR: ClamClient: could not connect to remote clam daemon, Couldn't connect to server
ERROR: Clamonacc: daemon is local, but a connection could not be established
There is a potentially better way we can solve this though by specifically checking for the “clamd” socket as a dependency when we try to start the systemd clamonacc.service.
Here is our updated systemd unit file:
# /etc/systemd/system/clamonacc.service
[Unit]
Description=ClamAV On Access Scanner
Requires=clamav-daemon.service
After=clamav-daemon.service syslog.target network.target
[Service]
Type=simple
User=root
ExecStartPre=/bin/bash -c "while [ ! -S /var/run/clamav/clamd.ctl ]; do sleep 1; done"
ExecStart=/usr/bin/clamonacc -F --config-file=/etc/clamav/clamd.conf --log=/var/log/clamav/clamonacc.log --move=/root/quarantine
[Install]
WantedBy=multi-user.target
Save the above to a file located @ /etc/systemd/system/clamonacc.service
.
Do not enable and start the service just yet.
3. Testing clamonacc functionality
Let’s manually test clamonacc, we can do so by running the following command to launch the daemon and have it fork to the background.
sudo clamonacc --config-file=/etc/clamav/clamd.conf --log=/var/log/clamav/clamonacc.log --move=/root/quarantine
Verify it started successfully:
ubuntu@ip-172-31-81-67:~$ sudo tail /var/log/clamav/clamonacc.log
--------------------------------------
ClamInotif: watching '/home' (and all sub-directories)
ClamInotif: watching '/var/www' (and all sub-directories)
Let’s try downloading a virus test file to see if it detects it automatically and quarantines it.
sudo mkdir /var/www/html/testfolder
sudo chown ubuntu /var/www/html/testfolder
cd /var/www/html/testfolder
wget www.eicar.org/download/eicar.com
ls -l
We should see that the file doesn’t exist in the folder we downloaded it to:
ubuntu@ip-172-31-81-67:/var/www/html/testfolder$ ls -l
total 0
What about in the quarantine folder?
ubuntu@ip-172-31-81-67:/var/www/html/testfolder$ sudo ls /root/quarantine
eicar.com
There it is.
How about the logs?
ubuntu@ip-172-31-81-67:/var/www/html/testfolder$ sudo tail /var/log/clamav/clamonacc.log
--------------------------------------
ClamInotif: watching '/home' (and all sub-directories)
ClamInotif: watching '/var/www' (and all sub-directories)
/var/www/html/testfolder/eicar.com: Win.Test.EICAR_HDB-1 FOUND
/var/www/html/testfolder/eicar.com: moved to '/root/quarantine/eicar.com'
Bingo.
Let’s kill our ad-hoc clamonacc now that we’re done testing.
sudo kill `ps auxf | grep clamonacc | grep -v "grep" | awk '{print $2}'`
3. Enable and start clamonacc daemon
sudo systemctl enable clamonacc
sudo systemctl start clamonacc
In Conclusion
This has been quite a long article to write, that said, when I was going through setting up and configuring ClamAV with On-Access Scanning recently — at the time I would have really appreciated if someone had consolidated all this knowledge and experience in one place.
ClamAV may not be the most efficient (see memory requirements) or versatile (see caveats about clamonacc) Antivirus solution around. However, for Linux it seems to be one of, if not, the best option in a very limited market.
What I’ve learned throughout this experience
The Linux ecosystem for full blown Antivirus solutions is very limited (most people seem to settle on a rookit scanner). ClamAV seems to be the universally adopted option, with that said, it has a ways to go if it wants to compete with the Antivirus options that exist for Windows. Specifically it needs to work towards:
- Reducing running memory footprint of it’s scanning engine (clamd)
- Correct a bug related to recursively monitoring directories that contain special files
- Handle the ability to scan files based on specific access operation (read, write, or execute).
- Improved testing / support by package maintainers and integration with systemd (for example clamav-daemon incorrectly reports start up state of clamd to systemd)
For those planning to use On-Access Scanning, my advice is
Don’t try to use it to scan the entire file system and don’t use OnAccessPrevention (unless latency is not an issue).
Use it on specific directories that are likely sources of malware based on your systems use-case (for example file server directories, home directories, or website writable directories).
Hopefully, this article was helpful and insightful for those who are looking to setup On-Access Scanning understand the caveats and steps necessary to get it running under Ubuntu 18.04 with ClamAV 0.102.3.