In order to detect short-lived processes we intercept new processes
executions as they happen, and cache them for later use.
When a new connection is established, then we check if the PID of the
connection is cached, and use the details of the process to ask the user
to allow or deny it.
However, there're some situations where the path or cmdline of a PID,
doesn't correspond with the one that's establishing the connection.
Given the same PID:
- Sometimes we receive from the tracepoint a wrong/non-existent path.
- Other times we receive a "helper" which is the one executing the
real binary that opens the connection.
For these reasons now when a new connection is established, we read the
path to the binary from proc. If the PID is cached and the cached path
differs, then we'll use the path from proc.
We lose a bit of performance, but hopefully we'll be more consistent
with what the user expect, while at the same time keeping intercepting
short-lived processes.
Downsides: for execveat() executions we won't display the original binary.
Closes#771
Whenever a process exits, we delete the corresponding entry from
cache.
But when a process executes a new process (sh -c ls), we receive an
exit event for the parent, while the child continues working with *the
same PID*. Sometimes we don't receive exit events for the child, so the
entry was never removed from cache.
We should properly detect the exits, but forthe time being, delete
expired processes from cache every minute.
Up until now we loaded the eBPF modules from /etc/opensnitchd.
However there has been some problems upgrading the modules to newer
versions with the deb packages, because every file under /etc/ is
treated as a conffile, and whenever a conffile changes it prompt you to
update it or not. Some users decided to no upgrade it, ending up with
eBPF modules incompatible with the new daemon.
https://www.debian.org/doc/manuals/maint-guide/dother.en.html#conffiles
On the other hand, the FHS dictates that /etc/ is for configuration
files, and /usr/lib for object files:
"/usr/lib includes object files and libraries. [21] On some systems,
it may also include internal binaries that are not intended to be
executed directly by users or shell scripts."
https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s06.html
So now, we look for the eBPF modules under /usr/local/lib/opensnitchd/ebpf/
or /usr/lib/opensnitchd/ebpf/, and as a last resort under
/etc/opensnitchd/
Up until now some error and warning messages were only logged out to the
system, not allowing the user know what was happening under the hood.
Now the following events are notified:
- eBPF related errors.
- netfilter queue errors.
- configuration errors.
WIP, we'll keep improving it and build new features on top of this one.
- Get cmdline arguments from kernel along with the absolute path to the
binary.
If the cmdline has more than 20 arguments, or one of the arguments is
longer than 256 bytes, get it from ProcFS.
- Improved stopping ebpf monitor method.
Improved process detections by monitoring new processes execution.
It allow us to know the path of a process before a socket is opened.
Closes#617
Other improvements:
- If we fail to retrieve the path of a process, then we'll use the comm
name of the connection/process.
- Better kernel connections detection.
- If debugfs is not loaded, we'll try to mount it, to allow to use
eBPF monitor method.
Future work (help wanted):
- Extract command line arguments from the kernel (sys_execve, or mm
struct).
- Monitor other functions (execveat, clone*, fork, etc).
- Send these events to the server (GUI), and display all the commands
an application has executed.
The eBPF cache is meant mainly for certain applications that
establish 2-4 new connections in under 1-2 seconds. Thus, a cache of 1
minute per item was too much, 10-20 seconds is enough.
Also, check old items every minute to keep the number of items low.
* Allow to intercept some kernel connections
Some connections are initiated from kernel space, like WireGuard
VPNs (#454), NFS or SMB connections (#502) and ip tunnels (#500).
Note: This feature is complete for x86_64, WIP for aarch64, and not supported for armhf and i386
https://github.com/evilsocket/opensnitch/pull/513#issuecomment-924400824
More information regarding this change: #493
- Fixed reloading process monitor method if the configuration changes on
disk. This can occur in two situations: 1) if it's changed from the
UI, 2) if the user changes it manually.
- Ensure that we don't crash if there's an error changing the
method and ebpf is active.
- When changing monitor method to ebpf and it fails to start, stop it
anyway. It helps cleaning up kprobes and avoiding the error
"cannot write...: file exists".
- Fixed multiple race conditions when using the cache of PIDs.
- Improved the chances to hit the cache of inodes, which helps to keep
down the times to get the PID of a connection to <= 30us.
These caches are mainly used when not using "ebpf" proc monitor method.
When enabling the eBPF monitor method we dump the active connections,
but in some cases there're no active connections, and because of this
we're failing enabling this monitor method.
If there're no connections established, netlink returns 0 entries. It's
not clear if it's an indication of error in some cases or the expected
result.
Either way:
- fail only if we're unable to load the eBPF module.
- dump TCP IPv6 connections only if IPv6 is enabled in the syste,-
It'd probably be a good idea to write a module and encapsulate all the
functionality of the fields in funcs(), to lock them properly
(get/set maps, etc).
TODO: replace monitorLocalAddress() by
netlink.AddrSubscribeWithoptions(), to receive addresses' events
asynchronously.
Sometimes when a new connection is about to be established, we don't get
the PID of the process using the eBPF proc monitor method. But in some
rare situations, the kernel still holds information about the connection
(sock_diag struct basically). We assume that these connections are
initiated from kernel space.
Per some debugging, this doesn't seem to be always the root cause, so
these connections will only be shown if InterceptUnknown config field is
set to true.
On systems that have been running for a long time (for example 552
days) we were failing parsing the starttime field:
```
Could not find or convert Starttime. This should never happen.
Please report this incident to the Opensnitch developers:
strconv.Atoi: parsing "4242026842": value out of range
```
- extra: fixed tests.
* Use ebpf program to find PID of new connections.
before running the branch you have to compile ebpf_prog/opensnitch.c
opensnitch.c is an eBPF program. Compilation requires getting kernel source.
cd opensnitch
wget https://github.com/torvalds/linux/archive/v5.8.tar.gz
tar -xf v5.8.tar.gz
patch linux-5.8/tools/lib/bpf/bpf_helpers.h < ebpf_prog/file.patch
cp ebpf_prog/opensnitch.c ebpf_prog/Makefile linux-5.8/samples/bpf
cd linux-5.8 && yes "" | make oldconfig && make prepare && make headers_install # (1 min)
cd samples/bpf && make
objdump -h opensnitch.o #you should see many section, number 1 should be called kprobe/tcp_v4_connect
llvm-strip -g opensnitch.o #remove debug info
sudo cp opensnitch.o /etc/opensnitchd
cd ../../../daemon
--opensnitchd expects to find opensnitch.o in /etc/opensnitchd/
--start opensnitchd with:
opensnitchd -rules-path /etc/opensnitchd/rules -process-monitor-method ebpf
Co-authored-by: themighty1 <you@example.com>
Co-authored-by: Gustavo Iñiguez Goia <gooffy1@gmail.com>
- don't clean cache by number of items.
- clean inodes from cache every 2' if the descriptor symlink doesn't exist
anymore, or if the lastSeen time is more than 5 minutes.
- launch cache cleaners before start a new process monitoring method,
and start it only once for the life time of the daemon.
- do not store in cache the Time objects, only the nanoseconds of
the last updated time.
- if the inode of a connection is found in cache, reorder the
descriptors to push the descritptor to the top of the list.
Also add cached the inode.
It turns out that when a new connection is about to be established,
when the process resolves the domain, the same inode is used to open the
tcp connection to the target. So if it's cached we save CPU cycles.
This also occurs when we block a connection and the process retries it,
or when a connection timeouts and the process retries it
(telnet 1.1.1.1).
- update the descriptors/inodes of a PID when it's found in cache.
- when a descriptor/inode is found in cache, push it to the top
of the descriptors list. The next time it's found in cache it'll be in
the 1st position of the list, saving CPU time.
- added test cases and benchmark helpers to help analyzing performance.
* maintain a cache of struct Process for currently active PIDs
decreases PID lookup time from ~100usec to ~5usec
* Update activepids.go
remove import "os"
Co-authored-by: themighty1 <you@example.com>
- De/Serialize IPv6 connections.
- Added SocketsDump() to list all sockets currently in the kernel.
- [proc details] Resolve all the sockets an application has opened
and translate them to network data, e.g:
```
ls -l /proc/1234/fd/
0 ... 25 -> socket[12345678]
```
to
```
0 .... 25 -> socket[12345678] - 54321:10.0.2.2 -> github.com:443,
state: established
```
New dialog added to display details of a process in realtime, gathered
from ProcFS.
Process tab -> double click on an app -> click on the button with the
search icon.
We have also improved the discovery of apps icons and names. It should
work better on systems where the DE is not properly configured.
Tested, but not bulletproof, still in beta.
we were not switching between process monitor methods properly, so we're
falling back to proc method in some cases.
Besides, there's seems to be a descriptors leaking problem in ftrace package
when closing resources.
