Page tree

Contents

We have written some scripts to set up the EdgeOS device with the correct firewall rules, to get your block lists and use the results to update the ipset sets that are created and to upload your firewall logs to us. You can download the scripts here, but you will probably find it easier to cut and paste the links in the instructions below.

Note:

  • EdgeOS is a fork of the Vyatta codebase. These instructions have been adapted from our Vyatta instructions but the background links below refer to Vyatta as the two are essentially the same.
  • If this is a new device, please allow up to 30 minutes for our systems to be updated.
  • If you are unclear about what the scripts will do you should read the background notes first. This also explains the difference between "Bridge" and "Router" mode.
  • We also have a PDF file showing how to setup a basic Vyatta/EdgeOS device and then apply ThreatSTOP to it.

Prerequisites

You should have a management station with a web browser that can read this page and an ssh client that can access the EdgeOS device.
You should have set up your EdgeOS device with the external ethernet port and ip address as well as a management ip address that can be accessed from your management station.
The EdgeOS device should have ssh enabled on it, which can be done by entering these commands from the console:

configure
set service ssh
commit
exit

Note:

You should also ensure that the device has a default gateway and name server configured. If you wish you may set the default name server of the device to one of the ThreatSTOP DNS servers (addresses below).

You should confirm that the management's ssh client can connect to the EdgeOS device and log in to it. Using that login session you should confirm that the EdgeOS device can also access/download files from the Internet. In particular you should check that your EdgeOS device can:

ping ftp.threatstop.com

It is recommended that you save the configuration prior to applying the ThreatSTOP changes, which can be done by entering the following from the console:

configure
save prethreatstop
exit

Note:

If this is a reinstallation you may wish to either rename the folder ts-vyatta something else or restore the system to its original state before running the reinstallation code.

You should also download the libnet-dns-perl module. This can be done by entering the following after logging in:

sudo apt-get install libnet-dns-perl

Setup

Copy and paste the following line into the EdgeOS ssh session:

curl ftp://ftp.threatstop.com/pub/ts-edgeos.tar.gz | tar xzv ; ts-vyatta/setup.pl --type r --blocklist <block list name>..threatstop.local --allowlist <allow list name>..threatstop.local

The device should download and unpack the ThreatSTOP scripts and then run the setup script which will display the following (if you have a sudo password enabled on your EdgeOS device then you will need to enter that when prompted).

===========================================================================
If you have not specified setup options on the command line then you will be
given the chance to specify them now. First time users running this by pasting
in the command from the ThreatSTOP website should probably not change anything
except the firewall names and start rule number if you have already created
some firewall rules.

On subsequent runs, probably the only things to change will be the block and
allow list ids. For each option, the default value is specified in [], just
press the ENTER key to accept it.

Note for the paranoid. The proposed changes to the Vyatta config, the changes
to /etc/rc.local, /etc/logrotate.d/messages and the new crontab are created in
the installation directory. If you choose not to allow this script to apply the
changes automatically then you can review them and then apply them manually.
===========================================================================

You will then be asked a number of questions and normally you should accept the default values. The critical questions are the third one asking about the external interface id and the subsequent ones concerning firewall rule names and numbers. If the external interface id is not eth0 then you must correct this. Likewise if you have existing firewall rules defined for the "in", "out" and "local" directions of the external interface you must enter their names and an appropriate rule number since otherwise those rules will no longer be applied to traffic on that interface. If you do have existing rules it is generally recommended that the ThreatSTOP rules are performed first and thus that they have the lowest numbers. The setup script creates the four ThreatSTOP rules with consecutive numbers. Thus, if your existing rules start at 10, you should specify that the script insert the rules starting from 1 to 5.

ThreatSTOP installation directory [/home/vyatta/ts-vyatta/]

ThreatSTOP ipset prefix [TS]

Install type routed - external interface id [eth0]

Firewall name for interface eth0 direction in: [TSrtinrule]

Insert ThreatSTOP rules beginning at number? [10]

Add default accept? (strongly recommended if you do not have other
rules for this firewall name, not otherwise)[y]

Firewall name for interface eth0 direction local: [TSrtlocalrule]

Insert ThreatSTOP rules beginning at number? [10]

Add default accept? (strongly recommended if you do not have other
rules for this firewall name, not otherwise)[y]

Firewall name for interface eth0 direction out: [TSrtoutrule]

Insert ThreatSTOP rules beginning at number? [10]

Add default accept? (strongly recommended if you do not have other
rules for this firewall name, not otherwise)[y]

ThreatSTOP block list:<block list name>..threatstop.local

ThreatSTOP allow list:<allow list name>..threatstop.local

dig command location [/home/vyatta/ts-vyatta/dig]

Logfile to upload [/var/log/messages]

URL for submitting logs [https://threatstop.com/cgi-bin/logupload.pl]

Once these questions have been answered the script will create the install directory and copy files if required, test whether the firewall can download the block list from ThreatSTOP's DNS server and create the EdgeOS firewall rules (but not apply them) and the modifications of the various files that are needed.

Initial set up complete, testing
/home/vyatta/ts-vyatta/dig +tcp -t ptr @192.124.129.42<block list name>..threatstop.local
Test successful
Creating bridging rules to configure.route.sh
Creating local copy of logrotate.d/messages file
Creating crontab file
Creating local copy of rc.local file

Now you are asked whether you wish to accept the changes. If you have a complex/non-standard configuration you may wish to say N at this point and examine the files that have been created.

Apply changes: /home/vyatta/ts-vyatta/configure.route.sh,
/etc/logroate.d/messages, /etc/rc.local, crontab (Y/N)[Y]

Merging /home/vyatta/ts-vyatta/configure.route.sh

Removing and clearing crontab for root
no crontab for root
no crontab for root
# Update the ThreatSTOP lists. Every 2 hours, 0 minutes after the hour
# (00:15, 02:15, 04:15, etc.)
0 */2 * * * /home/vyatta/ts-vyatta/ipsetget.pl
# Force a logrotate if the log is > 100k. Check every 5 minutes after
the hour
5 * * * * perl -e'exec q(logroate -f /etc/logrotate.d/messages) if (stat q(/var/log/messages))[7]>100000;'

Copying modified /etc/logrotate.d/messages
`/home/vyatta/ts-vyatta/logrotated.messages' ->
`/etc/logrotate.d/messages'
Copying modified /etc/rc.local
`/home/vyatta/ts-vyatta/rc.local' -> `/etc/rc.local'

Finally you have the choice to run the script to get the block list data for the first time:

Run script for first time? (Y/N) [Y]

After some seconds (depending on the blocklist size and the connectivity to our DNS servers this may take up to a minute) you should see a report of a successful first run and the script completes. Assuming this last step reports success, you have installed ThreatSTOP on your EdgeOS routing device.
You should verify that none of the changes have broken basic connectivity and, if there are no problems, you should save the configuration so that it is used whenever the device reboots. This can be done by entering from the console:

configure
save threatstoprouter
save
exit

To view the contents of the block and allow lists, you will need to run the "ipset" command:

sudo ipset -L

The output will most likely go by too fast to view. You can pipe it to less so you can page through the output:

sudo ipset -L | less

Updating EdgeOS

When updating EdgeOS by using the image upgrade procedure, the scripts will be moved to a different location on the filesystem. This results in the block list not being updated and logs not uploading. In order to get ThreatSTOP working again, you will need to re-run the setup procedure, but this time it is run in a update mode. The configuration will not be modified, but the cronjobs that update the block list and uploads the log are recreated.
To perform the update copy and paste the following line into the EdgeOS ssh session:

wget -O - ftp://ftp.threatstop.com/pub/ts-vyatta.tar.gz | tar xzv ; ts-vyatta/setup.pl --type u --blocklist<block list name>..threatstop.local --allowlist<allow list name>..threatstop.local

Troubleshooting

A guide to troubleshooting four common problems. If you are confused or if these steps do not help then please contact ThreatSTOP support.

  1. ThreatSTOP rules do not appear to block anything

    The most likely reason is that you have not correctly entered the firewall's IP address in the ThreatSTOP device definition page. You can verify whether the address the firewall uses is in our database by running the following from the command promt (SSH or console):

    wget -qO - http://www.threatstop.com/cgi-bin/validip.pl

    You should see a simple result stating the device's IP address and whether it is in the database or not. Database updates are not instantaneous but take place every 20 minutes; so, if you have recently added/modified the firewall IP details, you may wish to wait about half an hour before checking this. If there is no response at all then verify by using the ping command that the firewall can reach threatstop.com and, if not, that it can reach other places such as google.com. If you have no connectivity to ThreatSTOP but do have connectivity elsewhere please contact ThreatSTOP support for further information about the status of the ThreatSTOP infrastructure.

    If the address is valid but there are still problems you should manually run the blocklist retrieval procedure and then examine the output file. To download the blocklists do the following at the command prompt (SSH or console):

    sudo ts-vyatta/ipsetget.plless /tmp/tsoutput.txt

    In the output file the first few lines should look like:

    /home/vyatta/ts-vyatta/ipsetget.pl started at Mon Nov 1 10:49:01 2010
    /home/vyatta/ts-vyatta/dig +tcp t ptr @192.124.129.42<block list name>..threatstop.local

    ; <<>> DiG 9.4.3-P2 <<>> +tcp -t ptr @192.124.129.42<block list name>..threatstop.local
    ; (1 server found)
    ;; global options: printcmd
    ;; Got answer:
    ;; ->>HEADER<< opcode: QUERY, status: NOERROR, id: 32530
    ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 15, AUTHORITY: 2, ADDITIONAL:
    0

    If instead you see ;; connection timed out; no servers could be reached then there is a problem with either our DNS or with the internet path to us. If you confirm via pings etc. (see above) that the problem is with our DNS then please contact ThreatSTOP support for guidance on how to use a backup DNS server (and/or when the DNS will resume operation).

    If the status message is NXDOMAIN or REFUSED then the blocklist name is probably incorrect. Double-check that the name in the file is<block list name>..threatstop.local

    If the status is NOERROR but the ANSWER number is 0 (or possibly 1) then the likely issue is that you have forgotten to enable any blocklists. Go back to the devices page in your ThreatSTOP account, click edit by the appropriate entry and confirm that you have some blocklists enabled. If you are in standard mode normally you should have at least the "Basic", and "Advanced" blocklists enabled and probably either or both of "Botnets" and "Unix Server". In Advanced mode you should confirm that you have some lists checked and you should contact ThreatSTOP support to understand what the lists you have enabled should be blocking.

  2. ThreatSTOP blocks access to places it shouldn't

    Although ThreatSTOP tries very hard to ensure that we have zero false positives in our standard lists, we do occasionally miss something. If you are a community user or are using ThreatSTOP in standard mode on your firewall then please report the offending domain and ip address to ThreatSTOP support. Please check in the EdgeOS log file (/var/log/threatstop.log) that the IP address is indeed being blocked by the ThreatSTOP rules before contacting us.

    You can also add the ip address to a custom allow list

    Note:

    Our Advanced lists are not so well checked for false positives as we believe that it is up to each user to make a decision about the suitability of certain feeds. Some feeds - e.g. the "Parasites" feed - are known to block IP addresses that are not considered harmful by everyone. Do consider a custom allowlist (see above) as a first step and do please contact ThreatSTOP support to verify that the feeds you have chosen are appropriate to your situation.
  3. Other firewall rules do not appear to work

    The most common reason for this is that the ThreatSTOP setup script modified the existing firewall setup and in the process overwrote either the name of the firewall used for traffic or some of the rules - including changing the default action to accept.

    Take a look at the "prethreatstop" and "postthreatstop" saved configurations (both are in /opt/vyatta/etc/config) to see what changes have been made. One place to look at is in the interfaces section. If there are firewall names in the prethreatstop bridge interfaces then make sure that they remain there in the postthreatstop (check spelling differences). If those match then also check that you did not overwrite rules in that firewall name and also verify that if the firewall name had a "default-action accept" line previously.

  4. No Logs Uploaded

    In many cases logging issues are related to issue 1) above. If running the "validip" test described above is successful then you should verify that "log enable" is set appropriately in the current configuration (all ThreatSTOP firewall drop rules should log enable in them).

    You should also confirm that the systems root crontab and /etc/logrotate.d/messages files have been correctly modified. To check the root crontab do:

    sudo crontab -l

    If crontab is correctly setup there should be a line with a number, some asterisks and then the following perl command line:

    perl -e'exec q(/usr/sbin/logrotate -f /etc/logrotate.d/messages) if (stat q(/var/log/messages))[7]>100000;'

    To check the /etc/logrotate.d/messages enter:

    cat /etc/logrotate.d/messages

    If crontab is correctly setup there should be the following lines displayed:

    daily
    prerotate
    /home/ubnt/ts-vyatta/loguploadclient.pl
    endscript

    You should also be able to see the same lines in the relevant files in the ts-vyatta directory. An upload can be forcing a logrotation and then examining /tmp/tsoutput.txt for errors.

    sudo /usr/sbin/logrotate -f /etc/logrotate.d/messages

  5. Logging private addresses

    If private intranet addresses are being logged talking to other private addresses then it is possible that the issue is to do with the "log-martians enable" line that ThreatSTOP adds to the firewall configuration. You should verify that the traffic is indeed harmless and then remove that line.

Restore to previous state

If you have run setup and applied the changes and wish to return to the pre-threatstop configuration then you should perform the following command (assuming that you installed to /home/vyatta/ts-vyatta).
From a console enter:

ts-vyatta/revert.sh

Optionally you may also wish to remove the ts-vyatta directory (rm -r ts-vyatta/).
This has now restored all the files changed. To restore the configuration you should do the following:

configure
load presthreatstop
save
exit

It is possible that you may need to "load prethreatstop" more than once to handle "commit" errors. Once you have managed to load the old configuration without error you should probably reboot the EdgeOS device to be sure that it runs with no traces of ThreatSTOP changes in the system.