At the outset, I will state that, for my homebuilt firewall, I allow ONLY IPv4, and that is done by specifically disabling IPv6 support during GRUB using
GRUB_CMDLINE_LINUX_DEFAULT=" ... ipv6.disable=1 ..."
As part of my IPTABLES-based homebuilt firewall, I have scripts to manage various country-lists by category: BLK (blocklist), PRB (probationary, meaning watched), and WHT (whitelist).
In addition to those, I also manage manually created lists: BLK (blocklist-manual) and WHT (whitelist-manual).
The filtering for those lists, using IPSETs, is only on the two relevant chains:
At one point, I also had a script to optimize the rule sequencing based on traffic levels, higher traffic IPs being evaluated before the lesser traffic ones. The degree of complexity introduced, to manage that sequencing, led me to discontinue its use for fear of not “getting it right”.
My approach to the firewall was “modular” or “table-driven” in that my scripts would build the IPSET filtering based on configurations defined in a table, namely:
##################################################################################################
###
### $Id: FW_IPset_TARGETS_4.txt,v 1.3 2020/09/13 00:51:20 root Exp $
###
### Data table listing labels assigned to Non-Core CHAINS defined as TARGETS for IP sets that will be loaded into local IPTABLES firewall.
### Format: Leading '#' in column 1 suppresses the entry from being evaluated.
###
##################################################################################################
#f:NonCoreChain|CoreChain|Table| # relevant comments for clarifying usage of the TARGET
#
##################################################################################################
#
BOGONS_v4|PREROUTING|mangle|AFTER| # Load all CIDR related to published BOGONS, because used for spoofing
#
##################################################################################################
#
WHTmanual_v4|PREROUTING|mangle|AFTER| # Load all IP sets that track manually entered cases for WHITELISTed IPs
WHTmanual_v4|OUTPUT|mangle|AFTER| #
#
BLKmanual_v4|PREROUTING|mangle|AFTER| # Load all IP sets that track manually entered cases for BLACKLISTed IPs
BLKmanual_v4|OUTPUT|mangle|AFTER| #
#
##################################################################################################
#
WHTcountries_v4|PREROUTING|mangle|AFTER| # Load all IP sets related to WHITELISTed countries
WHTcountries_v4|OUTPUT|mangle|AFTER| #
#
PRBcountries_v4|PREROUTING|mangle|AFTER| # Load all IP sets related to PROBATIONed countries
PRBcountries_v4|OUTPUT|mangle|AFTER| #
#
BLKcountries_v4|PREROUTING|mangle|AFTER| # Load all IP sets related to BLACKLISTed countries
BLKcountries_v4|OUTPUT|mangle|AFTER| #
#
### END-of-LIST ###
The lists in that table are driven by the lists built from a second configuration table, for each “class” of actor/country list, namely:
##################################################################################################
###
### $Id: FW_IPset_Security_4_30.txt,v 1.4 2020/09/09 02:51:07 root Exp $
###
### Data table listing labels assigned to IP sets for use with local IPTABLES firewall. Contains default IPTABLES policy assignment for IPsets in grouping. Identifies TABLE/CoreChain for loading IP sets.
### Format: Leading '#' in column 1 suppresses the entry from being evaluated.
###
##################################################################################################
#
###|TABLE|mangle|###
###|cCHAIN|PREROUTING|###
###|DEFAULT|ACCEPT|###
###|SEQUENCE|AFTER|###
#
###|nCHAIN|WHTcountries_v4|###
#
#f:IPsetLabel|NonCoreChain|Table| # relevant comments for clarifying usage of the IP set
WHlist_Canada|WHTcountries_v4|mangle|
WHlist_USA|WHTcountries_v4|mangle|
WHlist_GB|WHTcountries_v4|mangle|
WHlist_France|WHTcountries_v4|mangle|
WHlist_Germany|WHTcountries_v4|mangle|
WHlist_Spain|WHTcountries_v4|mangle|
WHlist_Belgium|WHTcountries_v4|mangle|
WHlist_Netherlands|WHTcountries_v4|mangle|
WHlist_Denmark|WHTcountries_v4|mangle|
WHlist_Norway|WHTcountries_v4|mangle|
WHlist_Sweden|WHTcountries_v4|mangle|
WHlist_Austria|WHTcountries_v4|mangle|
WHlist_Switzerland|WHTcountries_v4|mangle|
WHlist_Australia|WHTcountries_v4|mangle|
WHlist_NewZealand|WHTcountries_v4|mangle|
WHlist_SouthAfrica|WHTcountries_v4|mangle|
WHlist_Greece|WHTcountries_v4|mangle|
#
### END-of-LIST ###
Those country-specific whitelists are identified and created based on their presence in yet another table specifying which countries have been whitelisted, namely:
####################################################################################################
###
### $Id: FW_IPset_INDEX_03_allowlist_countries.txt,v 1.1 2020/08/19 20:59:10 root Exp $
###
### Table providing local short/long form reference to 'country of origin' which are NOT deemed SUSPECT for IPTABLES packets ... UNTIL deemed otherwise.
###
####################################################################################################
#
### Probationary ALLOW countries
ca|Canada|ALL|
us|United States of America|USA|ALL
gb|United Kingdom|GB|ALL
fr|France|ALL
de|Germany|ALL
es|Spain|ALL
be|Belgium|ALL
nl|Netherlands|ALL
dk|Denmark|ALL
no|Norway|ALL
se|Sweden|ALL
at|Austria|ALL
ch|Switzerland|ALL
au|Australia|ALL
nz|New Zealand|NewZealand|ALL
za|South Africa|SouthAfrica|ALL
gr|Greece|ALL
#
#
#
### Deferred inclusion as ALLOW countries (see INDEX_TBDlist_countries.txt)
#in|India|ALL|
#kr|South Korea|SKorea|ALL|Republic of Korea #??? kr|KoreaSouth|SKorea|ALL|Republic of Korea
#mx|Mexico|ALL
#it|Italy|ALL
#ma|Morocco|ALL
#dz|Algeria|ALL
#ke|Kenya|ALL
#eg|Egypt|ALL
#tr|Turkey|ALL
#id|Indonesia|ALL
#th|Thailand|ALL
#
#il|Israel|ALL
#pl|Poland|ALL
#br|Brazil|ALL
#ar|Argentina|ALL
#jp|Japan|ALL
#tw|Taiwan|ALL
To give a sense of the scope of my firewall tools, here is the menu presented by my firewall admin script:
Select firewall administration option:
[C] Create firewall functionality FIREWALL__DoReset.sh
calls: - IPTABLES__PurgeUFW.sh
- IPTABLES__IPsets__PurgeAllSets.sh
- FIREWALL__PostFirstBootProfile_Desktop.sh
- IPSETS.sh
- IPTABLES__ShowStatus__Rules.sh
- IPTABLES__ShowStatus__IPsets.sh
[V] Verify firewall integrity FIREWALL__PostBootConfirmSanity.sh
calls: - FIREWALL__DoReset.sh
[U] Raise (up) firewall defenses FW_00__StructuredWrapper.sh
[D] Lower (down) firewall defenses FW_Admin__SetPromicuous.sh
[R] Restore saved firewall rules FW_Admin__IptablesRules_Restore.sh
[B] Save firewall rules, option for persistence FW_Admin__IptablesRules_Save.sh
[L] Display built-in IPTABLES rule report w/stats FW_Admin__IptablesRules_Show.sh
[RA] Report all firewall rules IPTABLES__ShowStatus.sh --force
[RT] Report differences in firewall statistics IPTABLES__ShowStatus.sh --diff
[SK] IP Sets - Rebuild complete IP set config IPSETS.sh
[SP] IP Sets - Purge all IP sets IPTABLES__IPsets__PurgeAllSets.sh
[SC] IP Sets - Create IPTABLES table entry IPTABLES__IPsets__CreateTableEntry.sh
[SL] IP Sets - Push list to load set IPTABLES__IPsets__PushListToLoad.sh
[SB] IP Sets - Save set definitions for persistence IPTABLES__IPsets__RestoreCreate.sh
[SR] IP Sets - Restore set definitions to KERNEL IPTABLES__IPsets__RestoreApply.sh
[SS] IP Sets - Show Status of IP Sets IPTABLES__ShowStatus__IPsets.sh
[K] Report kernel parameter values FW_Admin__ReportSysctlParameters.sh
Enter selection [C|V|U|D|R|B|L|RA|RT|SK|SP|SC|SL|SB|SR|SS|K] =>
Before anyone asks, I hesitate to share my firewall scripts … at this time … for the main reason that I don’t feel that they are “clean” enough, from the standpoint of embedded documentation explaining the structure, rationale, functionality, strategies and steps.
I hope to revisit those sometime this year, as part of my “migration” to the 26.04 release for UbuntuMATE.
As an example of what I mean by not “publication-ready”, here is one of the nearly two dozen scripts I have for managing various aspects of my firewall:
#!/bin/sh
#23456789+123456789+123456789+123456789+123456789+123456789+123456789+123456789+123456789+123456789+
####################################################################################################
###
### $Id: IPTABLES__ShowStatus__IPsets.sh,v 1.6 2020/09/26 19:13:30 root Exp $
###
### This script is intended to report on the status of expected IPTABLES FIREWALL components that relate to IP Sets.
###
####################################################################################################
DBG=0
###
### Debugging/Verbosity
### 1 = extra step-wise messaging
### 2 = additional logging for greater visibility on nature of packets which are dropped
### 3 = full-on messaging for greater visibility on actions being
###
#FUTURES: Add iptables rules.
###################################################################################################################
resetOrLoadIPset()
{
if [ -s ${TMP} ]
then
lastGroup=""
for ChainPair in `cat ${TMP} `
do
if [ ${DBG} -eq 3 ] ; then echo "ChainPair= ${ChainPair}" ; fi
group_IPset=`echo ${ChainPair} | cut -f1 -d \| `
if [ "${lastGroup}" != "${group_IPset}" ]
then
CustomChain=`echo ${ChainPair} | cut -f2 -d \| `
proceedResetOrLoad=1
if [ ${proceedResetOrLoad} -eq 1 ]
then
if [ ${DBG} -eq 3 ]
then
echo "DBG: Chains identified:"
cat ${TMP}
echo "\t group_IPset= ${group_IPset}"
echo "\t Hit return to continue ...\c" ; read k <&2 ; echo ""
fi >&2
case ${group_IPset} in
BKlist_* ) SetTypeLABEL="BLACKLIST" ;;
PBlist_* ) SetTypeLABEL="PROBATION LIST" ;;
WHlist_* ) SetTypeLABEL="WHITELIST" ;;
esac
if [ ${verbose} -eq 1 ]
then
echo "\n\t [${groupCount}] Reviewing ${CustomChain}/${TABLE} context for ${SetTypeLABEL} grouping of hosts '${group_IPset}' ... " >&2
fi
groupCount=`expr ${groupCount} - 1 `
IPsetEXISTS=`ipset list ${group_IPset} 2>>/dev/null | awk -v sName="${group_IPset}" 'BEGIN{
dum=0 ; }
{
if ( index($0,"Name:") == 1 ){
if( $2 != sName ){
print "CORRUPT: $2" ;
dum=1 ;
} ;
} ;
if ( dum == 0 && index($0,"Number of entries") == 1 ){
if( $2 == 0 ){
print "EMPTY" ;
dum=1 ;
}else{
print "POPULATED" ;
dum=1 ;
} ;
} ;
}END{
if ( dum == 0 ){
print "NO_SET" ;
} ;
}' `
doRebuild=0
### FUTURES: TBD keep this logic loop
if [ ${doReportIPsets} -eq 1 ]
then
reportIPset
#echo "doRebuild= ${doRebuild}"
fi
fi #proceedResetOrLoad
fi
done #ChainPair
else
### TBD ????
echo "\t\t Possible corrupt file. No valid reference identified for IP set in ${CustomChain}/${TABLE} context." >&2
fi
} #resetOrLoadIPset()
###################################################################################################################
define_IPsets()
{
###
### FUTURES: Add logic to verify existence of IP set load data files
### before processing ANY entries for a given ipSetList file
### possibly prompting to give override to proceed rather than abandon.
###
#FW_IPset_Security_4_01.txt ## Intended to be processed semi-automatically (in-scope)
#FW_IPset_Manual_BKlist_01_corp.txt ## Intended to be manually constructed (FUTURES)
echo "\n Reviewing condition of security configuration files ..."
rm -f ${TMP}.setConfigFiles
{ ls -1 ${Prod_LIB}/FW_IPset_Security_4_*.txt 2>>/dev/null
ls -1 ${Prod_LIB}/FW_IPset_Manual_4_*.txt
} >${TMP}.setConfigFiles 2>>/dev/null
if [ ! -s ${TMP}.setConfigFiles ]
then
echo "\t\t WARNING: No ipset configuration files found:"
ls -1 ${Prod_LIB}/FW_IPset_Security_4_*.txt
ls -1 ${Prod_LIB}/FW_IPset_Manual_4_*.txt
return
fi
for ipSetList in `cat ${TMP}.setConfigFiles `
do
if [ ! -s "${ipSetList}" ] ; then echo " *** ERROR -- Empty file:" ; ls -l "${ipSetList}" ; echo "" ; fi
done
echo "\n Will process per specifications in following security configuration files:\n"
cat ${TMP}.setConfigFiles | awk '{ printf("\t %s\n", $0 ) ; }'
for ipSetList in `cat ${TMP}.setConfigFiles `
do
if [ ${DBG} -eq 3 ] ; then echo "DBG: ipSetList: ${ipSetList}" ; fi
if [ ${doReportIPsets} -eq 1 ]
then
doOptIPset=1
else
doOptIPset=0
if [ ${doAll} -eq 1 ]
then
doOptIPset=1
else
echo "\n Do you wish to process IP sets for grouping '${ipSetList}' ? [y|N] => \c" ; read optIPset
if [ -z "${optIPset}" ] ; then optIPset="N" ; fi
case ${optIPset} in
y* | Y* ) doOptIPset=1 ;;
* ) echo "\t Skipping ..." ;;
esac
fi
fi
if [ ${doOptIPset} -eq 1 ]
then
### FUTURES: Sanity check on load point for IP Sets using parameters from '${ipSetList}' files.
### REFERENCE: '###|nCHAIN|BLKcountries_v4|###' Not used on ChainPair
#GroupingChain=`grep '|nCHAIN|' ${ipSetList} | cut -f3 -d\| `
#echo "\t GroupingChain= ${GroupingChain}"
### REFERENCE: '###|cCHAIN|PREROUTING|###' Not used on ChainPair
#CoreChain=`grep '|cCHAIN|' ${ipSetList} | cut -f3 -d\| `
#echo "\t CoreChain= ${CoreChain}"
### REFERENCE: '###|TABLE|raw|###'
TABLE=`grep '|TABLE|' ${ipSetList} | cut -f3 -d\| `
if [ -n "${TABLE}" ]
then
if [ ${global} -eq 1 ]
then
case ${CustomChain} in
BOGONS_v4 ) doCustom=1 ;;
* ) doCustom=0 ;;
esac
else
doCustom=1
fi
if [ ${doCustom} -eq 1 ]
then
echo "\n\n
######################################################################################################
START -- Creation/Loading of rules and data for IP sets for `basename ${ipSetList} `
######################################################################################################\n"
### REFERENCE: '###|DEFAULT|DROP|###'
IPset_DefaultAction=`grep '|DEFAULT|' ${ipSetList} | cut -f3 -d\| `
echo "\t\t IPset_DefaultAction= '${IPset_DefaultAction}' ..."
if [ ${DBG} -eq 3 ] ; then echo "DBG: table identified: ${TABLE}" ; fi
grep -v '^#' ${ipSetList} | awk '{ if ( NF > 0 ) print $1 }' >${TMP}.chains
if [ ${DBG} -eq 3 ] ; then awk '{ printf("\t || %s\n", $0 ) ; }' ${TMP}.chains ; fi
applyOutgoing=0
matchDir="src"
rm -f ${TMP}
grep "|${TABLE}|" ${TMP}.chains | cut -f1,2 -d\| | tac >${TMP}
groupCount=`wc -l ${TMP} | awk '{ print $1 }' `
exitCode=0
resetOrLoadIPset
if [ ${exitCode} -eq 0 ]
then
echo "\t\t Firewall integrity confirmed for all IP sets in this grouping."
else
exitCodeAll=${exitCode}
fi
if [ ${verbose} -eq 1 ] ; then echo "\t DONE -- for `basename ${ipSetList} `\n" ; fi
#ipset list -name | more
if [ ${DBG} -eq 3 ] ; then ipset list -terse | more ; fi
fi
fi
fi
done
} #define_IPsets()
###################################################################################################################
initializeVariablesIPTABLES()
{
if [ ${DBG} -eq 3 ] ; then echo "** initializeVariablesIPTABLES" >&2 ; fi
START=`date`
STRT=`pwd`
Profile=P1
TMP=/tmp/`basename "$0" ".sh" `.tmp
rm -f ${TMP}
Oasis=${Oasis:=/Oasis}
Prod_LIB="${Oasis}/bin"
echo " Prod_LIB= '${Prod_LIB}' ..."
## Renamed: 00__FW__BlockLists to IPsets_LIBRARY
IPsetLoadBatchDir="${Oasis}/IPsets_LOAD"
IPsetSaveBatchDir="${Oasis}/IPsets_SAVE"
#
#
LOGlevel=6
PacketSampler="-m limit --limit 1/minute"
#
IPv4=`which iptables `
if [ -z "${IPv4}" ] ; then echo "\n\t ERROR: Unable to locate 'iptables' in the environment PATH. Unable to proceed.\n Bye!\n" ; exit 1 ; fi
IPv6=`which ip6tables `
} #initializeVariablesIPTABLES()
###################################################################################################################
checkIPTABLESrules()
{
rm -f ${TMP}.rules.list
${IPv4} -t ${TABLE} -L ${CustomChain} | grep ${group_IPset} >${TMP}.rules.list
if [ ${DBG} -eq 3 ]
then
echo "CONTENTS of ${TMP}.rules.list :"
cat ${TMP}.rules.list
fi
missingIPsetRules=0
if [ ! -s ${TMP}.rules.list ]
then
echo "\t Expected IPTABLES rules are missing ... Need to rebuild FIREWALL ..."
missingIPsetRules=1
doRebuild=1
exitCode=1
fi
} #checkIPTABLESrules()
###################################################################################################################
reportIPset()
{
# Report status of IP set
if [ -z "${group_IPset}" ] ; then echo "\t group_IPset= {EMPTY}" ; return ; fi
if [ ${DBG} -eq 3 ] ; then echo "missingIPsetRules= ${missingIPsetRules} [something to show]" ; fi
if [ ${DBG} -eq 3 ] ; then echo "\n Items in ${IPsetSaveBatchDir}:\n" ; ls ${IPsetSaveBatchDir} ; fi
if [ ${DBG} -eq 3 ] ; then echo "group_IPset= ${group_IPset}" ; fi
ls ${IPsetSaveBatchDir}/ipset.${group_IPset}.save >${TMP}.dataload 2>>/dev/null
if [ -s ${TMP}.dataload ]
then
#if [ ${DBG} -ge 1 ]
if [ ${DBG} -eq 3 ]
then
echo "\t\t Next file to be used for group verification:"
cat ${TMP}.dataload | awk '{ printf("\t\t\t %s\n", $0 ) ; }'
echo "\t\t\t Hit return to continue ...\c" >&2; read k ; echo ""
fi >&2
for dataload in `cat ${TMP}.dataload `
do
if [ -s ${dataload} ]
then
batch_IPset=`basename ${dataload} | cut -f2 -d\. `
#batch_Persist=`basename ${dataload} | cut -f3 -d\. `
if [ ${verbose} -eq 1 ]
then
echo "\t\t Verifying existences of IP set '${batch_IPset}' ..."
fi
if [ ${DBG} -eq 3 ] ; then echo "DO: ipset list ${batch_IPset} --terse" ; fi
rm -f ${TMP}.${batch_IPset}
ipset list ${batch_IPset} --terse >${TMP}.${batch_IPset} 2>&1
if [ $? -eq 0 ]
then
countRefL=`grep '^References' ${TMP}.${batch_IPset} | cut -f2 -d\: | awk '{ print $1 }' `
countLoad=`grep '^Number of entries' ${TMP}.${batch_IPset} | cut -f2 -d\: | awk '{ print $1 }' `
countSave=`wc -l ${dataload} | awk '{ print $1 }' ` ; countSave=`expr ${countSave} - 1 `
if [ "${countLoad}" -eq "${countSave}" ]
then
if [ ${countRefL} -lt 2 ]
then
echo "\t\t\t Expected count of '${batch_IPset}' rules [2] not loaded in IPTABLES [${countRefL}] ..."
exitCode=1
else
if [ ${verbose} -eq 1 ]
then
#References: 2
#Number of entries: 4831
awk '{ if( index($0,"References") == 1 ){ print $0 } ; if( index($0,"Number of entries") == 1 ){ print $0 } ; }' ${TMP}.${batch_IPset} | awk '{ printf("\t\t\t %s\n", $0 ) ; }'
fi
checkIPTABLESrules
fi
else
echo "\t\t\t Mismatch between '${batch_IPset}' items in memory [${countLoad}] and backup file [${countSave}] ..."
cat ${TMP}.${batch_IPset} | awk '{ printf("\t\t\t %s\n", $0 ) ; }'
exitCode=1
fi
else
echo "\t\t RE-CREATE AND RELOAD ... 'ipset' facility does not recognize IP set: ${batch_IPset}"
exitCode=1
fi
rm -f ${TMP}.${batch_IPset}
else
echo "\t\t EMPTY: ${dataload} ..."
exitCode=1
fi
done
else
echo "\t\t Unable to locate required file: ${IPsetSaveBatchDir}/ipset.${group_IPset}.save"
exitCode=1
fi
} #reportIPset()
#################################################################################################################
#################################################################################################################
echo ""
while [ ! -d ${Oasis}/bin ] ; do sleep 10 ; done # provide mechanism to avoid failure at boot by allowing time for mount process to happen.
initializeVariablesIPTABLES
doReportIPsets=1
doAll=0
noprompt=0
rebuildSets=0
verbose=0
global=0
while [ $# -gt 0 ]
do
case $1 in
--global ) global=1 ; shift ;;
--verbose ) verbose=1 ; shift ;;
--doall ) doAll=1 ; shift ;;
--sanity ) doReportIPsets=1 ; shift ;;
--noprompt ) noprompt=1 ; shift ;;
--rebuildsets ) rebuildSets=1 ; shift ;;
* ) echo "\n\t Invalid option '$1' used on command line. \n\t Allowed: [ --sanity | --noprompt | --rebuildsets ] . \n Bye!\n" ; exit 1 ;;
esac
done
exitCodeAll=0
define_IPsets
echo "START = ${START}"
echo " END = `date`"
echo "\n DONE. [`basename $0 `]\n"
if [ ${exitCodeAll} -ne 0 ]
then
exit ${exitCodeAll}
fi
exit 0
exit 0
exit 0
