The Pi-hole FTL engine
Go to file
DL6ER 514a770abc Add .github/PULL_REQUEST_TEMPLATE.md (#24) 2017-05-01 14:44:03 +02:00
.github Add .github/PULL_REQUEST_TEMPLATE.md (#24) 2017-05-01 14:44:03 +02:00
obj Ignore all files in obj directory (#5) 2017-02-16 21:50:13 +01:00
.gitignore Main .gitignore (#6) 2017-02-16 21:56:16 +01:00
.travis.yml Travis matrix build configuration (#7) 2017-02-19 14:00:08 +01:00
FTL.h Add audit log implementation in FTL (extension to already existing >top-domains and >top-ads commands, use ">top-ads for audit") 2017-04-22 01:05:11 +02:00
FTLtest.sh Minor change to FTLtest.sh 2017-03-12 11:02:20 +01:00
LICENSE Initial commit (adding license) 2017-02-02 12:37:54 +01:00
Makefile Remove necessity to use libmath routines 2017-04-14 18:59:39 +02:00
README.md Rearrange debugging instructions 2017-04-11 11:14:33 +02:00
args.c Some non-POSIX systems use different conventions for exit status values. 2017-03-29 14:00:51 +02:00
config.c Simplified config logic 2017-04-07 15:21:12 +02:00
daemon.c Get PID only once 2017-04-14 18:07:54 +02:00
flush.c Don't handle clients[] and forwarded[] data structure different than the other structs 2017-04-14 00:18:22 +02:00
gc.c Bugfix: Correct overTime data during GC runs if chached queries have been found 2017-04-20 15:32:43 +02:00
grep.c Close file without checking for the pointer (we return early when we cannot open the file) 2017-04-22 01:18:54 +02:00
log.c Remove necessity to use libmath routines 2017-04-14 18:59:39 +02:00
main.c Skip the analysis of too old queries in the logs altogether. By this, we can skip the initial GC run, since there will be nothing to be collected 2017-04-20 16:01:21 +02:00
parser.c Ignore log entries with "<name unprintable>" during log parsing. Fixes #22 2017-05-01 13:07:43 +02:00
request.c Send whether a domain is blocked due to a wildcard rule when asked for audit data 2017-04-22 16:15:35 +02:00
routines.h Use getpid() when saving PID to file 2017-04-14 17:59:01 +02:00
setupVars.c Increase maximum length of parsed lines in setupVars.conf to 100000 2017-04-14 14:00:02 +02:00
signals.c Try to detect who send SIGTERM/SIGINT signals 2017-04-12 15:01:39 +02:00
socket.c bzero is obsolete. Using memset instead 2017-04-08 23:46:33 +02:00
structs.c Add audit log implementation in FTL (extension to already existing >top-domains and >top-ads commands, use ">top-ads for audit") 2017-04-22 01:05:11 +02:00
threads.c Removed logg_const_str 2017-04-08 18:36:30 +02:00

README.md

Pi-hole FTL

(c) 2017 Pi-hole, LLC (https://pi-hole.net)

This project is copyright under the latest version of the EUPL.

Please see LICENSE file for your rights under this license.

Codacy Badge


Compatibility list

Board Tested OS CPU architecture Suitable binaries Tested by
VirtualBox Ubuntu 16.10 amd64 linux-x86_64 @DL6ER
Raspberry Pi Zero Raspbian Jessie armv6l arm-linux-gnueabi @DanSchaper
Raspberry Pi 1 Raspbian Jessie armv6 arm-linux-gnueabi @DL6ER
Raspberry Pi 2 Raspbian Jessie armv7l arm-linux-gnueabihf and arm-linux-gnuabi @TechnicalPyro
Raspberry Pi 3 Raspbian Jessie armv7l arm-linux-gnuabi and arm-linux-gnueabihf @DL6ER
Raspberry Pi 3 openSUSE aarch64 aarch64-linux-gnu @DL6ER
NanoPi NEO armbian Ubuntu 16.04 armv7l arm-linux-gnueabihf @DL6ER
Odroid-C2 Ubuntu 16.04 aarch64 aarch64-linux-gnu @DanSchaper
C.H.I.P Debian armv7l arm-linux-gnueabihf @Promofaux
OrangePi Zero armbian Ubuntu 16.04 armv7l arm-linux-gnueabihf @DL6ER

If your device is not listed you can get your CPU architecture by running lscpu. Download some binaries and try which one work. If you want to add a new device, open an issue or create a PR for the README.


How to test FTL?

FTL is now part of the Pi-hole development branches. If you want to test it, use pihole checkout dev

How to compile FTL from source?

  1. Clone the repo
  2. make
  3. sudo make install
  4. sudo service pihole-FTL start

Debugging pihole-FTL

Debugging instructions (when FTL is not starting reliably)

Once you are used to it, you can skip most of the steps and debugging is actually quite easy and gives you insights into how software (not limited to pihole-FTL) works.

  1. Install screen and gdb (probably sudo apt-get install screen gdb)
  2. Start a screen session (it will allow you to come back even if the SSH connection died)
  • If you don't know about screen, then read about it (you will love it!)
  1. Start a screen session using screen
  2. Ensure that pihole-FTL is terminated (e.g. sudo killall pihole-FTL)
  3. Arrange file permissions to be able to start FTL as your current user (the following assumes you are logged in as user pi).
  • sudo touch /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port
  • sudo chown pi:pi /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port
  • sudo chmod 0644 /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port
  1. Start pihole-FTL in the debugger: gdb pihole-FTL
  2. Type run debug to start FTL (you should see some lines of text and FTL should start successfully).
  3. You can now close the terminal (Ctrl+A and then D to detach) and come back later using (screen -r) when it has crashed
  4. If it has crashed, copy&paste the terminal output, and
  5. type also backtrace and post the output in a (new) issue
  6. We might ask for additional information in order to isolate your particular issue

Simplified debugging instructions (when FTL is running)

FTL has been designed such that a debugger can be attached to an already running process to ease debugging. Use sudo gdb -p $(cat /var/run/pihole-FTL.pid) to attach to the already running pihole-FTL process. You can leave off sudo if you are running pihole-FTL with the current user. Once loading of the symbols has finished (the (gdb) input prompt is shown), run continue to continue operation of pihole-FTL inside the debugger. All debugger features are now available.

If pihole-FTL has crashed, copy&paste the terminal output into a (new) issue. Also type backtrace and include its output. We might ask for additional information in order to isolate your particular issue.

When you want to detach the debugger from FTL without terminating the process, you can hit Ctrl+C and enter detach followed by quit.

Command line arguments

  • debug - Don't go into daemon mode (stay in foreground) + more verbose logging
  • test - Start FTL and process everything, but shut down immediately afterwards
  • version - Don't start FTL, show only version

Command line arguments can be arbitrarily combined, e.g. pihole-FTL debug test

File locations

  • /var/log/pihole-FTL.log log file
  • /var/run/pihole-FTL.pid PID file
  • /var/run/pihole-FTL.port file containing port on which FTL is listening

Socket connections

connect via e.g. telnet 127.0.0.1 4711 port may be automatically incremented if 4711 isn't available

Implemented keywords (starting with >, subject to change):

  • >quit: Closes connection to client

  • >kill: Terminates FTL

  • >stats : Get current statistics

domains_being_blocked 19977
dns_queries_today 104749
ads_blocked_today 279
ads_percentage_today 1.396606
  • >overTime : over time data (10 min intervals)
0 163 0
1 154 1
2 164 0
3 167 0
4 151 0
5 143 0
6 171 0
7 147 0
[...]
  • >top-domains : get top domains
0 8462 x.y.z.de
1 236 safebrowsing-cache.google.com
2 116 pi.hole
3 109 z.y.x.de
4 93 safebrowsing.google.com
5 96 plus.google.com
[...]

Variant: >top-domains (15) to show (up to) 15 entries

  • >top-ads : get top ad domains
0 8 googleads.g.doubleclick.net
1 6 www.googleadservices.com
2 1 cdn.mxpnl.com
3 1 collector.githubapp.com
4 1 www.googletagmanager.com
5 1 s.zkcdn.net
[...]

Variant: >top-ads (14) to show (up to) 14 entries

  • top-clients : get top clients (IP addresses + host names (if available))
0 9373 192.168.2.1 router
1 484 192.168.2.2 work-machine
2 8 127.0.0.1 localhost

Variant: >top-clients (9) to show (up to) 9 client entries

  • >forward-dest : get forward destinations (IP addresses + host names (if available))
0 12940 1.2.3.4 some.dns.de
1 629 5.6.7.8 some.other.dns.com
  • >querytypes : get collected query types
A (IPv4): 7729
AAAA (IPv6): 5880
PTR: 12
SRV: 0
  • >getallqueries : get all queries that FTL has in its database
1483964292 IPv4 apis.google.com 1.2.3.4 3
1483964293 IPv4 clients5.google.com 1.2.3.4 2
1483964294 IPv4 lh3.googleusercontent.com 1.2.3.4 2
1483964295 IPv4 ogs.google.com 1.2.3.4 2
1483964297 IPv4 plus.google.com 1.2.3.4 2
1483964299 IPv4 www.google.com 2.3.4.5 2
1483964302 IPv4 www.googleadservices.com 2.3.4.5 1
1483964312 IPv4 www.gstatic.com 2.3.4.5 2
1483964333 IPv4 www.linguee.de 2.3.4.5 2

Variant: >getallqueries (37) show (up to) 37 latest entries

  • >getallqueries-time 1483964295 1483964312 : get all queries that FTL has in its database in a limited time interval
1483964295 IPv4 ogs.google.com 1.2.3.4 2
1483964297 IPv4 plus.google.com 1.2.3.4 2
1483964299 IPv4 www.google.com 2.3.4.5 2
1483964302 IPv4 www.googleadservices.com 2.3.4.5 1
1483964312 IPv4 www.gstatic.com 2.3.4.5 2

Variant: >getallqueries-time 1483964295 1483964312 (17) show matches in the (up to) 17 latest entries. Note that this does not necessarily mean that 15 entries will be returned

  • >getallqueries-domain www.google.com : get all queries that FTL has in its database for a specific domain name
1483964299 IPv4 www.google.com 2.3.4.5 2

Variant: >getallqueries-domain www.google.com (15) (see notes above)

  • >getallqueries-client 2.3.4.5 : get all queries that FTL has in its database for a specific client name or IP
1483964299 IPv4 www.google.com 2.3.4.5 2
1483964302 IPv4 www.googleadservices.com 2.3.4.5 1
1483964312 IPv4 www.gstatic.com 2.3.4.5 2
1483964333 IPv4 www.linguee.de 2.3.4.5 2

Variant: >getallqueries-client 2.3.4.5 (12) (see notes above)

  • >recentBlocked : get most recently pi-holed domain name
www.googleadservices.com

Variant: >recentBlocked (4) show the four most recent vlocked domains

  • >memory : get information about FTL's memory usage due to its internal data structure
memory allocated for internal data structure: 178688 bytes (178.69 KB)
dynamically allocated allocated memory used for strings: 15084 bytes (15.08 KB)
Sum: 193772 bytes (193.77 KB)
  • >clientID : Get ID of currently connected client
6
  • >version : Get version information of the currently running FTL instance
version v1.6-3-g106498d-dirty
tag v1.6
branch master
date 2017-03-26 13:10:43 +0200