Monday, July 28, 2008

squid proxy server - mini howto

Squid proxy servers are a great way to control your inter browsing for your company. In a previous post I have mentioned all the good reasons why you should get one, and in this post I will try and show you and explain to you where and how to do this.

You need to edit the main configuration file located under /etc/squid/squid.conf. In this file the following options must / can be used:

http_port 8080 - This is where you specify what port the proxy server must listen on. If you go to your Internet explorer to set your proxy and proxy port, this is the port number.

cache_mem 64 MB - This line specify the amount of memory that squid can use for caching. There are many theories out there how to calculate, but a good understanding could be obtained from the following criteria: ( from http://www.devshed.com/ )

x=Size of cache dir in KB (i.e. 6GB=~6,000,000KB) y=Average object size
(just use 13KB z=Number of directories per first level directory
(((x / y) / 256) / 256) * 2 = # of directories
As an example, I use 6GB of each of my 13GB drives, so:
6,000,000 / 13 = 461538.5 / 256 = 1802.9 / 256 = 7 * 2 = 14
So my cache_dir line would look like this:
cache_dir 6000 14 256

cache_swap_low 80 - When you set your cache_dir to lets say 2048 Mb or 2 GB, then this value is the low mark for squid to stop rotating or clean out the cache.

cache_swap_high 90 - Same as explanation above, but this will tell squid when to start free up old cache when it gets to the set level, and stop when it gets to the percentage of disk size ( 80 percent ).

maximum_object_size 1024 KB - This will tell squid not to cache any objects or files larger than 1 MB. It depends on how big the squid server is and how fast you want your cache to be, but also the amount of disk space you have, because you might fill up the space before you know it !

minimum_object_size 0 KB - This specify what the minimum size of a file or object is allowed to be, set this to 0 if you don't want to specify anything.

maximum_object_size_in_memory 64 KB - This specify the size of an object or file that is allowed to be placed in cache_mem amount of memory, Normally used for fast accees type files for browsing, don't make too high, it will hog the memory pool.

cache_dir ufs /var/spool/squid 2048 256 256 - This is where you specify what the disk space or disk space usage is for squid cache on your server. The cache_swap_high and cache_swap_low will look at this value and know when to rotate the cache when it hits the 90 an 80 percent mark.

cache_access_log /var/log/squid/access.log - This will tell squid where to write the access details to. You will run your reporting software on this file to see who browsed where on the Internet, amount of bandwidth used etc. You can use the below script file to concatenate the log files each month, so you have a directory will monthly access.log files in them :

#!/bin/bash
############################################################################## #Copyright : LDS - http://www.lds.za.net/
#
# Variables : ARCHIVE - directory where the SQUID files are archived
# : LOGDIREC - location
# : SQUIDLOG - SQUID log file name
##############################################################################
ARCHIVE=/var/log/archive/squid
LOGDIREC=/var/log/squid
SQUIDLOG=access.logrm $ARCHIVE/$SQUIDLOG -f
cp -dp $LOGDIREC/$SQUIDLOG $ARCHIVE/$SQUIDLOG
cp /dev/null $LOGDIREC/$SQUIDLOG
cd $ARCHIVE
mv ./`date +'%Y%m'`.gz ./temp.gz
gunzip ./temp.gz
(cat ./temp $SQUIDLOG gzip > ./`date +'%Y%m'`.gz) && rm temp mv temp
failed-`date +'%Y%m%d'`
#
# END OF SCRIPT
##############

ftp_user squid@lds.za.net - This specify the default username to send to Anonymous FTP sites.

auth_param basic program /usr/lib/squid/smb_auth -W CORE -U
The above line to to send authentication to a Microsoft based authentication server ( normally a domain controller )

auth_param basic children 8
auth_param basic realm AOSL Proxy Server
auth_param basic credentialsttl 2 hours

The above line sets options for the Authentication module.

client_lifetime 1 hour - This option prevent open connections to hog the squid process ( Internet Explorer browser left open on a PC )

half_closed_clients off - This works in conjunction with the above line to kill inactive connections

shutdown_lifetime 3 seconds - This option is to speed up the shutdown time when you stop squid.

acl password proxy_auth REQUIRED - This option work with the authentication options you have set, this will force authentication when a request comes through.

The below options is to have control over which Internal IP's can do anything, which sites are blocked etc.

acl openip src "/etc/squid/openip.cfg"
acl badsites dstdomain "/etc/squid/badsites.cfg"
acl opensites dstdomain "/etc/squid/opensites.cfg"
acl restricted_sites dstdomain "/etc/squid/restricted_sites"
acl restricted_users proxy_auth "/etc/squid/restricted_users"
acl priv_sites dstdomain "/etc/squid/priv_sites"
acl priv_users proxy_auth "/etc/squid/priv_users"
acl BONYUSERS dst 160.254.119.0/24

From the file names at the end ( which you should create ) explains what the file contain and what it will do for you.

The below lines is to "activate" the above lines

http_access allow BONYUSERS
http_access allow openip
http_access allow opensites
http_access allow restricted_users restricted_sites
http_access deny restricted_users
http_access allow priv_users priv_sites
http_access deny badsites
http_access allow password

The rest of the configuration files is not much needed, but the short explanation for that config line is given inside the /etc/squid directory.

As stated earlier, please post a comment to get more help.

Friday, July 25, 2008

Sendmail support - sendmailreload script

I have made this simple script you can use for your mail server when you need to activate any changes you have made to any of the config files inside the /et/mail directory. I know I have made a posting before with the script in it, but this one is a bit better, and this post is dedicated to it ;-)

Here is the script:

#!/bin/bash
MAILDIR="/etc/mail"
cd /etc/mail
makemap hash $MAILDIR/virtusertable.db < $MAILDIR/virtusertable
makemap hash $MAILDIR/mailertable.db < $MAILDIR/mailertable
makemap hash $MAILDIR/access.db < $MAILDIR/access
makemap hash $MAILDIR/aliases.db < $MAILDIR/aliases
makemap hash $MAILDIR/domaintable.db < $MAILDIR/domaintable
newaliases > /dev/null 2> /dev/null
wait
ps auxw grep sendmail grep accepting awk '{print "kill -HUP "$2}' sh
echo "Rebuild aliases run now - "`date` >> /var/log/maillog


The reason why it is better to use this script is that when you start to get a very busy mail server, with huge amount of mail in the mail queue, you don't want to restart the sendmail service everytime you made a change, rather run this script.

Sendmail support - aliases file

Sendmail use this file to "alias" email to users, groups, script files or even Mailmanagers.
With a standard Linux installation, this file is for some reason located in the /etc directory and not in the /etc/mail directory like the rest of the email configuration files.

I believe it is best to keep all the email configuration files in one directory, so I suggest you MOVE the /etc/aliases and the /etc/aliases.db files into the /etc/mail directory. Once you have done this, you MUST edit the /etc/mail/sendmail.cf file and change the location where sedmail is looking for the alias file. so you need to edit /etc/mail/sedmail.cf and look for the line:

" O AliasFile=/etc/aliases "

Change this line to read the following:

"O AliasFile=/etc/mail/aliases"

Write and quit the file, and again do a " service sendmail reload " or just run that sendmailreload script I have posted previously.

Lets look at the contents and the uses for the aliases file.

Option 1:

If you need to forward mail to more than one mailbox, example is sales, then you would use the aliases file. Inside the virtusertable file, you will state that sales@lds.za.net goes to user account lin001.

Then in the aliases file, you will have the following line to forward the mail to 4 other users as well, even external mail account, the following is the exact line:

lin001: user1, user2, user3, bill@gates.com
or
sales@lds.za.net: user1, user2, user3, bill@gates.com

Option 2:

If you need to forward mail for a specific user to himself and someone else, because he is on leave, then you would do the following.

jdoe@lds.za.net: \jdoe, user2

The \ in front of the username prevents the mail from looping that that user. Since the virtusertable already relay the mail to that user, you will create a loop of mail to that user by sending it to them again in the alias file.

Option 3:

You need to run a script file when someone sends an email to your server, almost like list managers, but normally used to send someone a legal disclaimer or something. The following line in the aliases file must be used.

disclaimer@lds.za.net: /path/to/script/file

Option 4:

The MD is going oversee on a business trip, and wants his email forwarded somewhere else, or to the branch where he is going, do the following.

ltrovald@lds.za.net: ltrovald@usa.lds.za.net

This will forward all mail to another mail address as specified.

Option 5:

The MD wants his mail to be kept on the server AND sent to the other email address as well, then do the following:

ltrovald@lds.za.net: \ltrovald, ltrovald@usa.lds.za.net


Remember to write a quit the /etc/mail/aliases file, and run the following command before reloading sendmail

makemap hash /etc/mail/aliases.db < /etc/mail/access
service sendmail reload

That is it, please post comments if you need more help.

Thursday, July 24, 2008

Sendmail support - mailertable file

Sendmail use this file to forward mail for a domain itself as a whole to another host on the Internet or inside your network. Let say you have an internal exchange server, and the Linux server acts as a gateway on the internet, you can send all your email to the Linux server ( MX records ) and from there forward the mail internal to your exchange server.

The mailertable file can also be used to send an entire domain to one local user by doing:

domainname.com local:username

The virtusertable file can do the same as the above, by adding a line in the virtusertable file like:

@domainname.com username

The syntax to use in the mailertable file to send a domain email to another host, is done with the following example:

domainname.com smtp:[internalserver.domain.com]

That is it, again like always, send me your comments or questions so we can make this better for you !

Sendmail support - virtusertable file

Sendmail use this file to do mail routing between all the domains that is on your mail server. How do you think sendmail cope with all the support@ or sales@ email addresses for all the domains. This file and the aliases file are used to group and relay the email addresses to the correct user on the mail server. There is 5 ways to distributing the mail in this file, which I will show examples for ;-)

Edit this file by typing : " vi /etc/mail/virtusertable "

Inside this file, you need to do the config as follow:

# BEGIN

webmaster@lds.za.net ldsuser1
smith@lds.za.net jsmith
gbates@lds.za.net ldsuser2

# Domain linuxhelp.za.net

webmaster@linuxhelp.za.net lin001
smith@linuxhelp.za.net lin002
gbates@linuxhelp.za.net lin003

# END

As you can see from this example file, we tried to show that you can have exactly the same email address or name for each domain name, and this shows you how you route it to the different users on the system. to create a email user on a Linux server, use the following command, we will use the username lin001 as example:

" useradd -d /home/lin001 -s /bin/false -c "Linuxhelp mail user" -g mail lin001 "

This will create a user without bash access, so this is a safe way of adding mail users. The -d option is the home directory location, this will NOT move the pop3 mailbox from /var/spool/mail to /home/lin001.

The -s option specify the shell access, the reason why we use /bin/false is to prevent a mail user with a simple password to have bash access which will lead to security problems.

The -c option is to add a comment line in the /etc/passwd file to identify a username, this works great to find any suspicious users that could have been created by a hacker

again, just reload sendmail after you have made the changes, or use the sendmailreload script I have made.

Sendmail support - local-host-names file

This file is very simple, and does nothing else besides specifying what domains names is hosted on the server. Any domain name found in this file, will expect to find entries in the virtusertable file as well. If you have only one domain on this server, then you will have your domain in this file, and in the relay-domains file, and only users set up in the /etc/passwd file.

simply edit the file and add your domain name by issuing the command:

" vi /etc/mail/virtusertable"

Inside this file , add only your domain name like

lds.za.net
google.com
linuxhelp.za.net

write and save the file, and that is it. Add all the domain names you add here into the /etc/mail/relay-domains file as well. you will see that by default the relay-domains file does not exist, so just create it ;-)

That is all I can say about that file ;-)

Sendmail Support and Installation

My favorite SMTP mail program of all time is Sendmail. This was obviously the first Linux SMTP MTA that I have used, and I still do. There are many more MTA's out there, but this one is mine ;-)


Ok, once you have installed a brand new Linux server, the standard sendmail installation is secure enough not to allow anyone or any other network besides the localhost to send any email. You will see when you do a " netstat -anop grep LIST " that sendmail is running on port 25 on 127.0.0.1. This is perfect since in the old day people will forget that they have not secured their sendmail installations, and become a SPAM host for the Internet.


SO the first step is to change the listen IP from 127.0.0.1 to either 0.0.0.0 or your IP of your server. To do this, you need to edit the sendmail.cf and make the following changes:


" vi /etc/mail/sendmail.cf "

Inside this file, look for the line that looks like this:


"O DaemonPortOptions=Port=smtp,Addr=127.0.0.1, Name=MTA"


Now, what you need to do it to change the MTA agent to listen on 0.0.0.0 and not just the localhost ip 127.0.0.1, so change the line to:


"O DaemonPortOptions=Port=smtp,Addr=0.0.0.0, Name=MTA"


Once you have done this, create a file called " relay-domains " in the /etc/mail directory, and in this file you need to put your domain name and the first 3 octets of your IP range like example 192.168.0 , just that


Then you can issue the following command to activate your changes:


" service sendmail reload "


You will see that the MAT now listen on 0.0.0.0:25 and by adding your domain name and IP range to that relay-domains file, you will be able to start using your mail server.


Below is a script I have written which you can use to activate new changes " on-the'fly" without having to restart sendmail. If you have a sendmail server you use in an ISP environment, you don't want to restart sendmail all the time, you will run into issues. See script below:

#!/bin/bash
cd /etc/mail
makemap hash virtusertable.db < virtusertable
makemap hash mailertable.db < mailertable
makemap hash access.db < access
newaliases > /dev/null 2> /dev/null
wait
ps auxw grep sendmail grep accepting awk '{print "kill -HUP "$2}' sh
echo "Rebuild aliases run now - "`date` >> /var/log/maillog

Call this file “sendmailreload” or something, make the file executable by typing " chmod 755 sendmailreload" and place the file under /usr/sbin.

I will add more pages for sendmail from here, each page dedicated to each file sendmail uses like local-host-names , mailertable etc.

Wednesday, July 23, 2008

Securing your company with Linux firewalls

If you do not have a firewall for your company as yet ... well just get one now ! The Internet is just not the safe place it used to be you know;-)

Firewall do 1000 % more than they used to do in the good old day. Firewall are used as VOIP servers, VPN servers, VPN end-tunnel-servers, hosting servers, Authentication servers, RAS server etc etc.

The first reason to get a firewall is to protect your internal computer network from the outside world. Your data can be stolen, company information can be stolen, financial documents can be stolen, identify theft or even your customer data can be stolen, and if that comes out, pick your favourite holiday destination and flee the country.

You have different kind of Attacks on the Internet , too many to mention, but the most well known types are DoS attacked, port scans, sniffers, eavedrops, bots, irc clients, Spam host, open relays etc. By securing your network and ensuring that you keep track of any exploits on software ( you can have a secure firewall, but if your firewall allowed port 80 traffic, and your IIS server is exploitable, your firewall is useless to this attack). If you are still running a Bind version prior to July 2008 release, it is time to upgrade.

Your normal services that a firewall will not close down, is normally port 80 ( web server ) port 53 ( dns server ) port 25 ( smtp mail server ) port 110,143 ( email services ). While you can have all the best rules for your firewall, but your firewall is hosting your DNS as well, and some hacker exploits your bind software, they will take FULL control over your server. Now think about this, give a hacker full access to your Firewall, and give him a couple of hours, and you will have the worst business day ever. SO maybe it is time that you show the threats to you Financial clowns, and tell them to always give IT the lowest budget, and explain that a firewall should do nothing else besides firewalling. If you had your DNS separate from your Firewall, the risk is 100 % less.

Get some comments coming in, and I will give some more advice or rules even if you need.

Saving money with Proxy servers

OK ... here in South Africa we still do not have the right people in charge of our Telecommunications department (ICASA), so we are one of the most expensive broadband countries in the world!

Keeping this in mind, we need to find ways to ensure we use the small caps we have per month to the best way we know. Now this can be done in numerous ways, all with the help of a Linux server you should add between your router and your network.

The main things to look at that will save you money, is to implement a Linux Proxy server to do the following things for you:

1.) Set up a browsing report system, where you can pull a report each month to see exactly who those clowns on the network is that keep downloading heaps of data which is not work related. This will have an immediate impact on saving , since they now know you are watching them ... very closely...

2.) Set up a proper caching server to cache all the bandwidth eating things like videos, pictures and file on frequently accessed sites. This will prevent the situation where each user accessing cnn.com fetching files from the site directly. many files will be drawn from the cache directory on your server, saving you on bandwidth usage! You will see the amount of cached data when you run a report mention in step one.

3.) Set up authentication for browsing. This is a must to have, since each user must log on to browse, which will eliminate those users or kids or friends that is not allowed on your company network in the first place !. This authentication can be done on the server itself with a simple htaccess file or you can authenticate from your AD even. To maintain this solution is as easy as hating Robert Mugabe. Period.

4.) Install a content management program or options in your Squid proxy server. What you can achieve with this is simple, add a list of sites you want to deny access to like video sites, mp3 sites, blogging sites ( not this one ;-) or any other sites you know is not work related, and obviously no need for anyone to go there while at the office. I LOVE Youtube, but having Youtube open on the company network is just asking to fill up your ADSL cap in 3 days time. Block sites like facebook as well, time spent on these sites is as scary as seeing Amy Whinehouse these days.

5.) Implement a bandwidth management device or simple use delay pools that is build in squid proxy server. What this will do is prevent users actually browsing at top speeds. Let say you have a 4MB ADSL line but you only want to let your users browse at a maximum of a 1 MB ADSL line, you can do it with delay pools. On a happy business hours day, you can download at 75 K /sec which will hammer your line. By limiting all browsing users to a 1 MB line, the obvious saving will happen.

Now if you need any additional information, let me know.

Dedicated Linux Help and support

Well well... It is about time that I make this available to the outside world ! As you know ( I hope ) Linux is free for all, and I am sure it will for ever stay that way. The problem is no everyone knows how to use this beast, so in the spirit of Linux being Open Source, I will make my knowledge Open source as well, by posting it here.

I will try and cover as much of Advance Linux (Server environment) as I can, and hopefully you can use this as a good platform to work from. I am an IT Manager by profession, and I am the Head of IT outsourcing as well. I have installed, maintained and created hundreds of Linux network in my lifetime, all of them still running.

It is all about installing Linux right the first time, take no short cuts and SECURE your server, since there are so VERY advance Linux users out there ( normally 12 years old ) that can't wait to hack your box, and use it as a bot server, spam, rootkit server etc. If you visit here often, tell all your friends, still will hopefully grow to a very useful Linux support site ! Till later !