/usr/local/psa/admin/sbin/mail_auth_view
Tuesday 30 October 2012
Plesk – Get email passwords using the command line
There is a built in command to get the email/passwords in plesk. When you ssh to the server just use the following:
Cisco – VPN troubleshooting commands
show crypto isakmp
show vpn-sessiondb summary
clear crypto isakmp sa
show vpn-sessiondb summary
clear crypto isakmp sa
mailenable – tool to analyse the logs for errors
If you are having issues with MailEnable and need an easy way to analyse the logs (notepad doesn’t really cut it).
http://www.mailenable.com/tools/MeLogCheck.exe
This tool sumarizes the amount of errors of a type and then provides a description of the error and when it occured.
http://www.mailenable.com/tools/MeLogCheck.exe
This tool sumarizes the amount of errors of a type and then provides a description of the error and when it occured.
Friday 7 September 2012
Install spf records for all accounts on cPanel server
Command to install spf record on single cPanel account.
# /usr/local/cpanel/bin/spf_installer cPanel_username
Put actual username in place of cPanel_username
Instead of installing spf records on cPanel accounts one by one use the following script to install it in one step:
# cd /var/cpanel/users
# for i in `ls /var/cpanel/users` ;do /usr/local/cpanel/bin/spf_installer $i ;done
# /usr/local/cpanel/bin/spf_installer cPanel_username
Put actual username in place of cPanel_username
Instead of installing spf records on cPanel accounts one by one use the following script to install it in one step:
# cd /var/cpanel/users
# for i in `ls /var/cpanel/users` ;do /usr/local/cpanel/bin/spf_installer $i ;done
Tuesday 28 August 2012
Error 1721 - Plesk uninstall
To resolve this issue, perform the steps in the "Reregister the Windows Installer service" section. If you experience the issue after you perform these steps, you may also have to perform the steps in the "Run System File Checker" section.
Reregister the Windows Installer service
The Windows Installer service manages the installation and removal of programs. The service applies a set of centrally defined setup rules during the installation process. These setup rules define the installation and configuration of the installed program. Windows Installer uses the Microsoft installation engine to install or remove programs. If the registration of the Microsoft installation engine is corrupted, you may not be able to remove programs that you have installed by using Windows Installer. In that case, you have to unregister and reregister the Windows Installer service. To do this, follow these steps:
- Start your computer in Safe mode. To do this, follow these steps:
- Restart the computer.
- After the BIOS information is displayed, press F8.
- Use the DOWN ARROW key to select Safe Mode, and then press ENTER.
- Use the UP ARROW key and the DOWN ARROW key to select your computer, and then press ENTER.
- Log on to the computer as local administrator.
- Click Start, click Run, type cmd.exe, and then click OK.
- At the command prompt, type msiexec /unregister , and then press ENTER.
- At the command prompt, type msiexec /regserver, and then press ENTER.
- At the command prompt, type exit, and then press ENTER to close the command prompt.
- Remove the software program from Add or Remove Programs.
Run System File Checker
The System File Checker (Sfc.exe) utility is used for scanning protected operating system files to verify their version and integrity. If System File Checker detects any operating system file with the incorrect file version, it replaces the corrupted file with a file that has the correct version from the Windows installation source files.
To use System File Checker, follow these steps:
- Click Start, click Run, type cmd.exe, and then click OK.
- At the command prompt, type sfc /purgecache, and then press ENTER.
Note You may be prompted to provide Windows installation source files when you run the sfc /purgecachecommand. If the command is completed successfully, you will receive the following message:
Windows File Protection successfully made the requested change. - At the command prompt, type sfc /scannow, and then press ENTER.
Note This command may take several minutes to finish. You may also be prompted to provide Windows installation source files when you run the sfc /scannow command.
- At the command prompt, type exit, and then press ENTER to close the command prompt.
- Remove the software program from Add or Remove Programs.
Monday 13 August 2012
How do I install Suhosin under different Linux Distributions ? (RHEL / CentOS / Fedora)
A number of open source application are preferred widely across the web, hence its security becomes a vital concern to the developers, providers and more importantly the users. Inorder to protect the applications developed in PHP, Suhosin is strongly recommended. The Suhosin patch offers great help with protecting the PHP based application from being completely exploited. The patch is considered to offer an advanced protection system for PHP installations. The main idea behind designing Suhosin was, to offer protection for servers against various attacks and other known issues in PHP.
Step 1 : Download latest version of Suhosin, enter the following command for that purpose
# cd /opt
# wget http://download.suhosin.org/suhosin-0.9.32.1.tgz
NOTE : You must ensure that php-devel is installed:
# yum install php-devel
Step 2 : Then Compile Suhosin under PHP and RHEL / CentOS Linux using the below commands
# cd suhosin-0.9.32.1
# phpize
#./configure
# make
# make install
Step 3 : The Configure Suhosin onto the server, using the below command a configuration file of Suhosin would be created
# echo ‘extension=suhosin.so’ > /etc/php.d/suhosin.ini
Step 4 : Using the below command you must not restart the server
# service httpd restart
NOTE : Incase you have lighttpd activated on the server, you must use the below command to restart it
# service lighttpd restart
Step 5 : Now you must recheck whether Suhosin has been installed on the server. Using the below command you may do so
$ php -v
NOTE : You may run the following command to check for more details
<?php
phpinfo();
?>
Step 1 : Download latest version of Suhosin, enter the following command for that purpose
# cd /opt
# wget http://download.suhosin.org/suhosin-0.9.32.1.tgz
NOTE : You must ensure that php-devel is installed:
# yum install php-devel
Step 2 : Then Compile Suhosin under PHP and RHEL / CentOS Linux using the below commands
# cd suhosin-0.9.32.1
# phpize
#./configure
# make
# make install
Step 3 : The Configure Suhosin onto the server, using the below command a configuration file of Suhosin would be created
# echo ‘extension=suhosin.so’ > /etc/php.d/suhosin.ini
Step 4 : Using the below command you must not restart the server
# service httpd restart
NOTE : Incase you have lighttpd activated on the server, you must use the below command to restart it
# service lighttpd restart
Step 5 : Now you must recheck whether Suhosin has been installed on the server. Using the below command you may do so
$ php -v
NOTE : You may run the following command to check for more details
<?php
phpinfo();
?>
Sunday 12 August 2012
Enable passive mode in FTP server with CSF firewall
If you running a FTP server (Pureftp/Proftp) with a CSF firewall (very importantly in VPS or virtuozzo etc), it is very important to to enable passive mode, because this mode, works best for ftp clients protected by firewall since the client initiates the connection. If you dont enable this mode in VPS you might encounter problems like ftp server will be unresponsive or hang.
along with FTP server running Pure-ftp or Proftp, just follow the below steps…
I am posting here the README where FTP connection issues is stressed in CSF firewall documentation…
13. A note about FTP Connection Issues
######################################
It is important when using an SPI firewall to ensure FTP client applications
are configured to use Passive (PASV) mode connections to the server.
On servers running Monolithic kernels (e.g. VPS Virtuozzo/OpenVZ and custom
built kernels) ip_conntrack and ip_conntrack_ftp iptables kernel modules may
not be available or fully functional. If this happens, FTP passive mode (PASV)
won't work. In such circumstances you will have to open a hole in your firewall
and configure the FTP server to use that same hole.
For example, with pure-ftpd you could add the port range 30000:35000 to TCP_IN
and add the following line to /etc/pure-ftpd.conf and then restart pure-ftpd:
PassivePortRange 30000 35000
For example, with proftpd you could add the port range 30000:35000 to TCP_IN
and add the following line to /etc/proftpd.conf and then restart proftpd:
PassivePorts 30000 35000
FTP over SSL/TLS will usually fail when using an SPI firewall. This is because
of the way the FTP protocol established a connection between client and server.
iptables fails to establish a related connection when using FTP over SSL
because the FTP control connection is encrypted and so cannot track the
relationship between the connection and the allocation of an ephemeral port.
If you need to use FTP over SSL, you will have to open up a passive port block
in both csf and your FTP server configuration (see above).
Perversely, this makes your firewall less secure, while trying to make FTP
connections more secure.
How to enable passive mode?
1. Add Passive Port range 30000-350000 to your Pureftp or Proftp configuration file
(i) Pureftpd
open /etc/pure-ftpd.conf, and this line
PassivePortRange 30000 35000
(ii) ProFTP
Open /etc/proftpd.conf, and add this line
PassivePorts 30000 35000
2. Open the ports from 30000 – 35000 in your CSF firewall configuration file under TCP_IN
Open /etc/csf/csf.conf
# Allow incoming TCP ports
TCP_IN = "20,21,22,25,53,80,110,30000:35000"
Then restart firewall and ftp server.
service csf restart
service pureftpd restart (or)
service proftpd restart
Once this is done, open your ftp client and try connecting to ftp server. It should be able to work in passive mode.
along with FTP server running Pure-ftp or Proftp, just follow the below steps…
I am posting here the README where FTP connection issues is stressed in CSF firewall documentation…
13. A note about FTP Connection Issues
######################################
It is important when using an SPI firewall to ensure FTP client applications
are configured to use Passive (PASV) mode connections to the server.
On servers running Monolithic kernels (e.g. VPS Virtuozzo/OpenVZ and custom
built kernels) ip_conntrack and ip_conntrack_ftp iptables kernel modules may
not be available or fully functional. If this happens, FTP passive mode (PASV)
won't work. In such circumstances you will have to open a hole in your firewall
and configure the FTP server to use that same hole.
For example, with pure-ftpd you could add the port range 30000:35000 to TCP_IN
and add the following line to /etc/pure-ftpd.conf and then restart pure-ftpd:
PassivePortRange 30000 35000
For example, with proftpd you could add the port range 30000:35000 to TCP_IN
and add the following line to /etc/proftpd.conf and then restart proftpd:
PassivePorts 30000 35000
FTP over SSL/TLS will usually fail when using an SPI firewall. This is because
of the way the FTP protocol established a connection between client and server.
iptables fails to establish a related connection when using FTP over SSL
because the FTP control connection is encrypted and so cannot track the
relationship between the connection and the allocation of an ephemeral port.
If you need to use FTP over SSL, you will have to open up a passive port block
in both csf and your FTP server configuration (see above).
Perversely, this makes your firewall less secure, while trying to make FTP
connections more secure.
How to enable passive mode?
1. Add Passive Port range 30000-350000 to your Pureftp or Proftp configuration file
(i) Pureftpd
open /etc/pure-ftpd.conf, and this line
PassivePortRange 30000 35000
(ii) ProFTP
Open /etc/proftpd.conf, and add this line
PassivePorts 30000 35000
2. Open the ports from 30000 – 35000 in your CSF firewall configuration file under TCP_IN
Open /etc/csf/csf.conf
# Allow incoming TCP ports
TCP_IN = "20,21,22,25,53,80,110,30000:35000"
Then restart firewall and ftp server.
service csf restart
service pureftpd restart (or)
service proftpd restart
Once this is done, open your ftp client and try connecting to ftp server. It should be able to work in passive mode.
Saturday 4 August 2012
MySQL ERROR! The server quit without updating PID file
ERROR! The server quit without updating PID file
You may face this error while restarting MySQL service.
——————–
root@ [~]# /etc/init.d/mysql status
ERROR! MySQL is not running, but lock file (/var/lock/subsys/mysql) exists
——————–
If you are facing the above error just remove the MySQL lock file and restart the server. May be while restarting the service you will get the pid error.
root@ [~]# rm /var/lock/subsys/mysql
rm: remove regular empty file `/var/lock/subsys/mysql’? y
—————————
root@ [~]# /etc/init.d/mysql restart
ERROR! MySQL server PID file could not be found!
Starting MySQL… ERROR! The server quit without updating PID file (/var/lib/mysql/hostname.pid).
—————————
In this case check the MySQL process and kill it. In this case there is no more MySQL process, so I need to check the MySQL log.
———————
root@ [~]# ps aux | grep mysql
root 32616 0.0 0.0 107412 864 pts/3 S+ 11:14 0:00 grep mysql
——————–
If the issue still persist please check the MySQL log may be there is any incorrect entry on my.cnf file.
Sample log
—————
130728 11:23:45 [ERROR] /usr/sbin/mysqld: unknown variable ‘set-variable=max_user_connections=50?
130728 11:23:45 [ERROR] Aborting
130728 11:23:45 InnoDB: Starting shutdown…
130728 11:23:45 InnoDB: Shutdown completed; log sequence number 2153996
130728 11:23:45 [Note] /usr/sbin/mysqld: Shutdown complete
130728 11:23:45 mysqld_safe mysqld from pid file /var/lib/mysql/hostname.pid ended
—————
Edit the my.cnf file and #hash the enrty then start the MySQL.
—————
root@ [~]# /etc/init.d/mysql start
Starting MySQL.. SUCCESS!
root@ [~]#
You may face this error while restarting MySQL service.
——————–
root@ [~]# /etc/init.d/mysql status
ERROR! MySQL is not running, but lock file (/var/lock/subsys/mysql) exists
——————–
If you are facing the above error just remove the MySQL lock file and restart the server. May be while restarting the service you will get the pid error.
root@ [~]# rm /var/lock/subsys/mysql
rm: remove regular empty file `/var/lock/subsys/mysql’? y
—————————
root@ [~]# /etc/init.d/mysql restart
ERROR! MySQL server PID file could not be found!
Starting MySQL… ERROR! The server quit without updating PID file (/var/lib/mysql/hostname.pid).
—————————
In this case check the MySQL process and kill it. In this case there is no more MySQL process, so I need to check the MySQL log.
———————
root@ [~]# ps aux | grep mysql
root 32616 0.0 0.0 107412 864 pts/3 S+ 11:14 0:00 grep mysql
——————–
If the issue still persist please check the MySQL log may be there is any incorrect entry on my.cnf file.
Sample log
—————
130728 11:23:45 [ERROR] /usr/sbin/mysqld: unknown variable ‘set-variable=max_user_connections=50?
130728 11:23:45 [ERROR] Aborting
130728 11:23:45 InnoDB: Starting shutdown…
130728 11:23:45 InnoDB: Shutdown completed; log sequence number 2153996
130728 11:23:45 [Note] /usr/sbin/mysqld: Shutdown complete
130728 11:23:45 mysqld_safe mysqld from pid file /var/lib/mysql/hostname.pid ended
—————
Edit the my.cnf file and #hash the enrty then start the MySQL.
—————
root@ [~]# /etc/init.d/mysql start
Starting MySQL.. SUCCESS!
root@ [~]#
Wednesday 18 July 2012
Missing Dependency: perl(URI)
root@serverte [~]# yum install subversion
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* addons: centos.mirror.netriplex.com
* base: centos.mirror.netriplex.com
* extras: mirrors.igsobe.com
* updates: ftp.usf.edu
Excluding Packages in global exclude list
Finished
Setting up Install Process
Resolving Dependencies
–> Running transaction check
—> Package subversion.i386 0:1.6.11-7.el5_6.4 set to be updated
–> Processing Dependency: perl(URI) >= 1.17 for package: subversion
–> Finished Dependency Resolution
subversion-1.6.11-7.el5_6.4.i386 from updates has depsolving problems
–> Missing Dependency: perl(URI) >= 1.17 is needed by package subversion-1.6.11-7.el5_6.4.i386 (updates)
Packages skipped because of dependency problems:
subversion-1.6.11-7.el5_6.4.i386 from updates
Fix
wget http://mirror.centos.org/centos/5/os/i386/CentOS/perl-URI-1.35-3.noarch.rpm
rpm -Uvh perl-URI-1.35-3.noarch.rpm
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* addons: centos.mirror.netriplex.com
* base: centos.mirror.netriplex.com
* extras: mirrors.igsobe.com
* updates: ftp.usf.edu
Excluding Packages in global exclude list
Finished
Setting up Install Process
Resolving Dependencies
–> Running transaction check
—> Package subversion.i386 0:1.6.11-7.el5_6.4 set to be updated
–> Processing Dependency: perl(URI) >= 1.17 for package: subversion
–> Finished Dependency Resolution
subversion-1.6.11-7.el5_6.4.i386 from updates has depsolving problems
–> Missing Dependency: perl(URI) >= 1.17 is needed by package subversion-1.6.11-7.el5_6.4.i386 (updates)
Packages skipped because of dependency problems:
subversion-1.6.11-7.el5_6.4.i386 from updates
Fix
wget http://mirror.centos.org/centos/5/os/i386/CentOS/perl-URI-1.35-3.noarch.rpm
rpm -Uvh perl-URI-1.35-3.noarch.rpm
Installing git on centos 6
I’m installing git on centos 6.
Install dependencies:
Install MakeMaker:
Download and compile the latest git source:
Test installation by initializing a new git repository:
You should see “
Install dependencies:
yum -y install zlib-devel openssl-devel cpio expat-develgettext-devel gcc perl-ExtUtils-MakeMaker
Install MakeMaker:
cd /usr/local/src
wget http://files.directadmin.com/services/9.0/ExtUtils-MakeMaker-6.31.tar.gz
tar xvzf ExtUtils-MakeMaker-6.31.tar.gz
cd ExtUtils-MakeMaker-6.31
perl Makefile.PL
make
make install
Download and compile the latest git source:
cd /usr/local/src
wget http://git-core.googlecode.com/files/git-1.7.9.2.tar.gz
tar xvzf git-1.7.9.tar.gz
cd git-1.7.9.2
./configure
make
make install
Test installation by initializing a new git repository:
cd ~/
mkdir test.git
cd test.git
git init
You should see “
Initialized empty Git repository in /root/git-test/.git/” if the installation process was successful.
INSTALL GIT ON CENTOS CPANEL SERVER
I have a love/hate relationship with cpanel. On one hand it makes low end hosting a breeze, on the other its a huge pita if you want to do anything “outside the box”. Perhaps it’s not such a pain if you work with cpanel frequently. But I find cpanel beneficial because it aids in tasks that I don’t do frequently, and can offload many of those tasks to less technical people.
I was recently setting up a cpanel instance for someone and had the need to install git for version control. I configured the epel repository for use as usual and went to yum install git only to be met with a nasty error.
I was pretty surprised as I have installed git without issue before on CentOS boxes before. A bit of digging showed that installing cpanel actually made some modifications to my /etc/yum.conf.
So cpanel has blocked all perl packages from being installed or updated because they don’t want updates to break or conflict with their packages. Thankfully yum provides a nice one time workaround for this kind of situation.
So one command later and I now have git installed.
I was recently setting up a cpanel instance for someone and had the need to install git for version control. I configured the epel repository for use as usual and went to yum install git only to be met with a nasty error.
git-1.5.5.6-2.el5.i386 from epel has depsolving problems
--> Missing Dependency: perl(Error) is needed by package
git-1.5.5.6-2.el5.i386 (epel)
git-1.5.5.6-2.el5.i386 from epel has depsolving problems
--> Missing Dependency: perl-Git = 1.5.5.6-2.el5 is needed by package
git-1.5.5.6-2.el5.i386 (epel)
git-1.5.5.6-2.el5.i386 from epel has depsolving problems
--> Missing Dependency: perl(Git) is needed by package
git-1.5.5.6-2.el5.i386 (epel)
Error: Missing Dependency: perl(Git) is needed by package
git-1.5.5.6-2.el5.i386 (epel)
Error: Missing Dependency: perl(Error) is needed by package
git-1.5.5.6-2.el5.i386 (epel)
Error: Missing Dependency: perl-Git = 1.5.5.6-2.el5 is needed by package
git-1.5.5.6-2.el5.i386 (epel)
I was pretty surprised as I have installed git without issue before on CentOS boxes before. A bit of digging showed that installing cpanel actually made some modifications to my /etc/yum.conf.
exclude=apache* bind-chroot courier* dovecot* exim* httpd* mod_ssl* mysql* nsd* perl* php* proftpd* pure-ftpd* ruby* spamassassin* squirrelmail*
So cpanel has blocked all perl packages from being installed or updated because they don’t want updates to break or conflict with their packages. Thankfully yum provides a nice one time workaround for this kind of situation.
–disableexcludes=[repo]
disable exclude from main, for a repo or for
everything
So one command later and I now have git installed.
yum --disableexcludes=maininstallgit
Tuesday 17 July 2012
How to Install ffmpeg, ffmpeg-php and codecs automagically
A client recently requested ffmpeg to be installed on the server. I have done this in the past on a client's box but he had a different OS setup.
Everyone loves automagic installations! After the first failure I bumped into a forum that let's you download a tarball and install ffmpeg without any hitches and within 20 minutes. This attempt did work!
Pros and Cons of Auto Installer
Con: The downside to the ffmpeg auto install script, you end up downloading and installing many ffmpeg related libraries you might not need.
Pros: No errors and installtions works as expected (at least for my setup).
There are a few things you should know before you get started:
Must be a Linux/Unix/POSIX server
Must need root shell(bash) access
Must need 200 MB of free space in your server
The author of this gpl ffmpeg auto install script will know your server ip (see below how to remove this) upon installation.
Check if yum is installed or not in the server. If there is no yum, please make sure automake,autoconf and libtools are installed. It is reccommend to configure the distribution specific yum.
Setup Installation
First, open up SSH (I like using putty) and login as "root". Once there, run the below commands:
mkdir ~/ffmpeg-packages
cd ~/ffmpeg-packages
wget http://trulymanaged.com/ffmpeg/ffmpeginstaller.tar.gz
tar -xvzf ffmpeginstall.tar.gz
cd ffmpeginstall
Now let's begin the installation process
./install.sh
Once the installation starts you will see the verbose messages, letting you know the status of the installation.
Everyone loves automagic installations! After the first failure I bumped into a forum that let's you download a tarball and install ffmpeg without any hitches and within 20 minutes. This attempt did work!
Pros and Cons of Auto Installer
Con: The downside to the ffmpeg auto install script, you end up downloading and installing many ffmpeg related libraries you might not need.
Pros: No errors and installtions works as expected (at least for my setup).
There are a few things you should know before you get started:
Must be a Linux/Unix/POSIX server
Must need root shell(bash) access
Must need 200 MB of free space in your server
The author of this gpl ffmpeg auto install script will know your server ip (see below how to remove this) upon installation.
Check if yum is installed or not in the server. If there is no yum, please make sure automake,autoconf and libtools are installed. It is reccommend to configure the distribution specific yum.
Setup Installation
First, open up SSH (I like using putty) and login as "root". Once there, run the below commands:
mkdir ~/ffmpeg-packages
cd ~/ffmpeg-packages
wget http://trulymanaged.com/ffmpeg/ffmpeginstaller.tar.gz
tar -xvzf ffmpeginstall.tar.gz
cd ffmpeginstall
Now let's begin the installation process
./install.sh
Once the installation starts you will see the verbose messages, letting you know the status of the installation.
Sunday 15 July 2012
rkhunter software - Install Linux Rkhunter in RHEL, CentOS and Fedora
rkhunter (Rootkit Hunter) is a open source Unix/Linux based tool released under GPL that scans for rootkits, backdoors and possible local exploits. rkhunter is a shell script which carries out various checks on the local system to try and detect known rootkits and malware. It also performs checks to see if commands have been modified, if the system startup files have been modified, and various checks on the network interfaces, including checks for listening applications. It scans haddressden files, wrong permissions set on binaries, suspicious strings in kernel etc. To know more about Rkhunter and its features visit http://www.rootkit.nl/.
Installing Rkhunter (Rootkit Hunter) in RHEL, CentOS and Fedora
Step 1: Downloading Rkhunter
Login to your server if you enter via SSH as root and download the latest stable version of Rkhunter tool by going to http://www.rootkit.nl/projects/rootkit_hunter.html or use below Wget command to download it on your systems.
# cd /tmp
# wget http://ncu.dl.sourceforge.net/project/rkhunter/rkhunter/1.4.0/rkhunter-1.4.0.tar.gz
Step 2: Installing Rkhunter
Once you have downloaded the latest version, run the following commands as a root user to install it.
# tar -xvf rkhunter-1.4.0.tar.gz
# cd rkhunter-1.4.0
# ./installer.sh --layout default --install
Step 3: Updating Rkhunter
Run the RKH updater to fill the database properties by running the following command.
# /usr/local/bin/rkhunter --update
# /usr/local/bin/rkhunter --propupd
Step 4: Setting Cronjob and Email Alerts
Create a file called rkhunter.sh under /etc/cron.daily/, which then scans your file system every day and sends email notifications to your email address. Create following file with the help of your favourite editor.
# vi /etc/cron.daily/rkhunter.sh
Add the following lines of code to it and replace “YourServerNameHere” with your “Server Name” and “you@yourdomain.com” with your “Email address“.
#!/bin/sh
(
/usr/local/bin/rkhunter --versioncheck
/usr/local/bin/rkhunter --update
/usr/local/bin/rkhunter --cronjob --report-warnings-only
) | /bin/mail -s 'rkhunter Daily Run (PutYourServerNameHere)' you@yourdomain.com
Set execute permission on the file.
# chmod 755 /etc/cron.daily/rkhunter.sh
Step 5: Manual Scan and Usage
To scan the entire file system, run the Rkhunter as a root user.
# rkhunter --check
The above command generates log file under /var/log/rkhunter.log with the checks results made by Rkhunter. For more help or information please run the following command.
# rkhunter --help
Installing Rkhunter (Rootkit Hunter) in RHEL, CentOS and Fedora
Step 1: Downloading Rkhunter
Login to your server if you enter via SSH as root and download the latest stable version of Rkhunter tool by going to http://www.rootkit.nl/projects/rootkit_hunter.html or use below Wget command to download it on your systems.
# cd /tmp
# wget http://ncu.dl.sourceforge.net/project/rkhunter/rkhunter/1.4.0/rkhunter-1.4.0.tar.gz
Step 2: Installing Rkhunter
Once you have downloaded the latest version, run the following commands as a root user to install it.
# tar -xvf rkhunter-1.4.0.tar.gz
# cd rkhunter-1.4.0
# ./installer.sh --layout default --install
Step 3: Updating Rkhunter
Run the RKH updater to fill the database properties by running the following command.
# /usr/local/bin/rkhunter --update
# /usr/local/bin/rkhunter --propupd
Step 4: Setting Cronjob and Email Alerts
Create a file called rkhunter.sh under /etc/cron.daily/, which then scans your file system every day and sends email notifications to your email address. Create following file with the help of your favourite editor.
# vi /etc/cron.daily/rkhunter.sh
Add the following lines of code to it and replace “YourServerNameHere” with your “Server Name” and “you@yourdomain.com” with your “Email address“.
#!/bin/sh
(
/usr/local/bin/rkhunter --versioncheck
/usr/local/bin/rkhunter --update
/usr/local/bin/rkhunter --cronjob --report-warnings-only
) | /bin/mail -s 'rkhunter Daily Run (PutYourServerNameHere)' you@yourdomain.com
Set execute permission on the file.
# chmod 755 /etc/cron.daily/rkhunter.sh
Step 5: Manual Scan and Usage
To scan the entire file system, run the Rkhunter as a root user.
# rkhunter --check
The above command generates log file under /var/log/rkhunter.log with the checks results made by Rkhunter. For more help or information please run the following command.
# rkhunter --help
Thursday 12 July 2012
Differnt PHP Configurations
When it comes to php configuration, there are many options available. Users and administrators easily gets confused with this. Here is a small reference on some php configurations.
DSO – Provide this version of PHP via libphp4.so or libphp5.so (AKA mod_php). This is normally the fastest possible way to serve PHP requests, but PHP will execute as the user “nobody”. If both versions of PHP are available, it is impossible to configure both to be served as DSO unless the concurrent DSO patch was applied at build time.
SuPHP – Provide this version of PHP through mod_suphp. This is the most flexible way of serving PHP requests and tends to be very secure. PHP scripts are executed by the user who owns the VirtualHost serving the request.
FCGI – Provide this version of PHP through mod_fcgid. This is a very fast way of serving PHP requests, but php.conf will most likely require additional tuning to perform well. If Suexec is enabled, each user will create their own PHP FastCGI server automatically and PHP scripts will be executed by the user who owns the VirtualHost serving the request. If Suexec is disabled, the “nobody” user will own all of the PHP FastCGI server processes and PHP scripts will be executed by the “nobody” user. FCGI mode is recommended only for advanced administrators who understand how to tune the performance of mod_fcgid. Userdir requests will not function correctly with the basic mod_fcgid setup provided by cPanel.
CGI – Provide this version of PHP through mod_cgi or mod_cgid. If Suexec is enabled, PHP scripts will be executed by the user who owns the VirtualHost serving the request. If Suexec is disabled, the “nobody” user will execute all PHP scripts. Userdir requrests will not function correctly with the basic CGI setup provided by cPanel. It is intended as a fallback when the other preferred methods (DSO or SuPHP) are not available. Serving PHP as CGI is not particularly secure or fast regardless of whether Suexec is enabled.
Default PHP Version
This setting controls which version of PHP will be configured to handle the .php file extension. When both versions of PHP are enabled, .php4 will always be PHP4, .php5 will always be PHP5 and .php will be whichever is configured as the default. Overriding this setting in for individual VirtualHosts will be detailed below.
DSO Considerations
Apache directives like php_value and php_admin_value are features provided by libphp. This is the only setup where those are valid directives in .htaccess files or httpd.conf. When compiled with the concurrent DSO patch, php4_value, php5_value, etc should be used instead.
For PHP scripts to execute, permissions of 0644 are sufficient. The user “nobody” must have sufficient permissions to traverse to the PHP script and read it though. Any files created by PHP scripts will be created by the user “nobody”, so files and directories that will receive such output must be writable by that user.
SuPHP Considerations
EasyApache 3 compiles mod_suphp in paranoid mode with several patches to improve Apache userdir support. If you encounter problems with mod_suphp, please be aware that mod_suphp as shipped by cPanel behaves in very different ways than the pristine upstream version.
Apache directives like php_value are not valid for mod_suphp. It is possible to place a php.ini file in the directory containing the PHP script and specify these types of values in it.
NOTE: PHP does not merge the php.ini files together, so when a custom php.ini is used it must contain all of the required directives from the main php.ini file (for example, if Zend Optimizer is required, the new php.ini must load the extension.)
For PHP scripts to execute, permissions of 0400 are sufficient. Scripts are run as the user who owns the VirtualHost, and as long as this user has permissions sufficient to write to a file/directory, PHP scripts will also have the ability to do so. Mod_SuPHP performs various security checks before executing PHP scripts. Most can be disabled in Mod_SuPHP configuration file located at /opt/suphp/etc/suphp.conf
docroot – PHP scripts must reside under this directory. Default is /, change to /home/ to improve security.
allow_file_group_writable – prevents execution of PHP scripts with the group write bit set. Default is false. Changing to true will allow these scripts to execute but reduce security.
allow_file_others_writable – Same as above, others write bit.
allow_directory_group_writable – If you previously run PHP as DSO, you may have PHP scripts residing in a directory that is writable by group members. This security check prevents those scripts from executing. Default is false. Changing to true will allow these scripts to execute but reduce security.
allow_directory_others_writable – Same as above, others write bit.
check_vhost_docroot – This directive causes suphp to check that the target script resides in the document root of the VirtualHost serving a request. For a userdir request, SuPHP considers the domain part of the URL to be the VirtualHost serving the request. Setting this to true will cause userdir requests, and some types of PHP aliases to fail, though security will be significantly improved. Default it false.
userdir_overrides_usergroup – this is a configuration option allowed by cPanel specific patches. Setting this to true allows SuPHP to determine which user should execute a script from the userdir portion of the URL. Setting it to false will provide the normal mod_suphp behavior of executing scripts as the user specified by the domain portion of a URL.
paranoid_uid_check – Mod_SuPHP can be compiled with different security modes. EasyApache 3 uses the “paranoid” mode, though some scenarios are better served by the “force” mode. The paranoid UID check performed by mod_suphp verifies that the user ID that owns a script is the same one that is executing it. Setting this to false disables the UID check as in force mode.
paranoid_gid_check – Same as above, but with the group ID that owns the script. On some BSD systems it may be necessary to disable this check because files inherit the GID that owns the directory. Disabling this check should not significantly reduce security so long as the allow_file_group_writable and allow_directory_group_writable checks are left at their default values.
umask – The mod_suphp developers set this to a default of 0777 meaning that all permissions on files must be explicitly specified. This is a very secure setting, but causes many problems. The cPanel default is 0022. Use 0033, 0077, or 0777 for improved security.
min_uid – The lowest user ID that can own scripts being executed. Default value is 1. Set this value to 100 or 500 to greatly improve security. If you provide shared PHP scripts owned by root for your users, this value will need to be lowered to 0.
min_gid – The lowest group ID that can own scripts being executed. Default value is 1. Set this value to 100 or 500 to greatly improve security. If you provide shared PHP scripts owned by root for your users, this value will need to be lowered to 0.
full_php_process_display – When set to “true” mod_suphp will execute PHP scripts in a way that displays both the PHP interpreter and the SCRIPT_FILENAME in ps output. Setting this to “false” will improve security by hiding the SCRIPT_FILENAME.
The suphp.conf file includes a section called [phprc_paths] that can be used to override the standard handling of php.ini. To lock a particular PHP handler to its default php.ini file, simply uncomment the appropriate line under [phprc_paths]. There is also a configuration directive supported by mod_suphp in httpd.conf and .htaccess files called suPHP_ConfigPath that sets the path to the php.ini file. To prevent the use of this directive in .htaccess files, remove “Options” from the Apache AllowOverride setting. Note that the [phprc_paths] set in suphp.conf take precedence over any suPHP_ConfigPath settings.
FCGI Considerations
This is not a recommended configuration for PHP. It requires fine tuning of mod_fcgid to ensure that the server does not become overloaded with idle PHP processes.
Permissions of 0400 are sufficient to execute PHP scripts when running under Suexec. Permissions of 0444 are sufficient to execute PHP scripts when running with Suexec disabled.
With the standard cPanel FCGI configurations, the PHP binary is available as a URL in the VirtualHost. This should not be considered a very secure setup.
Userdir requests do not function with the FCGI setup provided by cPanel.
Apache directives like php_value are not valid for mod_fcgid. A custom php.ini file should be used instead as detailed in the SuPHP section above.
CGI Considerations
Like FCGI, this is not a recommended configuration for PHP. The PHP binary is available as a URL in the VirtualHost, and the setup is not very secure.
Permissions of 0400 are sufficient to execute PHP scripts when running under Suexec. Permissions of 0444 are sufficient to execute PHP scripts when running with Suexec disabled.
Several PHP options may prevent the CGI setup from functioning correctly, particularly DiscardPath and ForceCGIRedirect. If you are having trouble with this configuration, please verify these options are disabled.
Userdir requests do not function with the CGI setup provided by cPanel.
Apache directives like php_value are not valid for mod_cgi. A custom php.ini file should be used instead as detailed in the SuPHP section above.
DSO – Provide this version of PHP via libphp4.so or libphp5.so (AKA mod_php). This is normally the fastest possible way to serve PHP requests, but PHP will execute as the user “nobody”. If both versions of PHP are available, it is impossible to configure both to be served as DSO unless the concurrent DSO patch was applied at build time.
SuPHP – Provide this version of PHP through mod_suphp. This is the most flexible way of serving PHP requests and tends to be very secure. PHP scripts are executed by the user who owns the VirtualHost serving the request.
FCGI – Provide this version of PHP through mod_fcgid. This is a very fast way of serving PHP requests, but php.conf will most likely require additional tuning to perform well. If Suexec is enabled, each user will create their own PHP FastCGI server automatically and PHP scripts will be executed by the user who owns the VirtualHost serving the request. If Suexec is disabled, the “nobody” user will own all of the PHP FastCGI server processes and PHP scripts will be executed by the “nobody” user. FCGI mode is recommended only for advanced administrators who understand how to tune the performance of mod_fcgid. Userdir requests will not function correctly with the basic mod_fcgid setup provided by cPanel.
CGI – Provide this version of PHP through mod_cgi or mod_cgid. If Suexec is enabled, PHP scripts will be executed by the user who owns the VirtualHost serving the request. If Suexec is disabled, the “nobody” user will execute all PHP scripts. Userdir requrests will not function correctly with the basic CGI setup provided by cPanel. It is intended as a fallback when the other preferred methods (DSO or SuPHP) are not available. Serving PHP as CGI is not particularly secure or fast regardless of whether Suexec is enabled.
Default PHP Version
This setting controls which version of PHP will be configured to handle the .php file extension. When both versions of PHP are enabled, .php4 will always be PHP4, .php5 will always be PHP5 and .php will be whichever is configured as the default. Overriding this setting in for individual VirtualHosts will be detailed below.
DSO Considerations
Apache directives like php_value and php_admin_value are features provided by libphp. This is the only setup where those are valid directives in .htaccess files or httpd.conf. When compiled with the concurrent DSO patch, php4_value, php5_value, etc should be used instead.
For PHP scripts to execute, permissions of 0644 are sufficient. The user “nobody” must have sufficient permissions to traverse to the PHP script and read it though. Any files created by PHP scripts will be created by the user “nobody”, so files and directories that will receive such output must be writable by that user.
SuPHP Considerations
EasyApache 3 compiles mod_suphp in paranoid mode with several patches to improve Apache userdir support. If you encounter problems with mod_suphp, please be aware that mod_suphp as shipped by cPanel behaves in very different ways than the pristine upstream version.
Apache directives like php_value are not valid for mod_suphp. It is possible to place a php.ini file in the directory containing the PHP script and specify these types of values in it.
NOTE: PHP does not merge the php.ini files together, so when a custom php.ini is used it must contain all of the required directives from the main php.ini file (for example, if Zend Optimizer is required, the new php.ini must load the extension.)
For PHP scripts to execute, permissions of 0400 are sufficient. Scripts are run as the user who owns the VirtualHost, and as long as this user has permissions sufficient to write to a file/directory, PHP scripts will also have the ability to do so. Mod_SuPHP performs various security checks before executing PHP scripts. Most can be disabled in Mod_SuPHP configuration file located at /opt/suphp/etc/suphp.conf
docroot – PHP scripts must reside under this directory. Default is /, change to /home/ to improve security.
allow_file_group_writable – prevents execution of PHP scripts with the group write bit set. Default is false. Changing to true will allow these scripts to execute but reduce security.
allow_file_others_writable – Same as above, others write bit.
allow_directory_group_writable – If you previously run PHP as DSO, you may have PHP scripts residing in a directory that is writable by group members. This security check prevents those scripts from executing. Default is false. Changing to true will allow these scripts to execute but reduce security.
allow_directory_others_writable – Same as above, others write bit.
check_vhost_docroot – This directive causes suphp to check that the target script resides in the document root of the VirtualHost serving a request. For a userdir request, SuPHP considers the domain part of the URL to be the VirtualHost serving the request. Setting this to true will cause userdir requests, and some types of PHP aliases to fail, though security will be significantly improved. Default it false.
userdir_overrides_usergroup – this is a configuration option allowed by cPanel specific patches. Setting this to true allows SuPHP to determine which user should execute a script from the userdir portion of the URL. Setting it to false will provide the normal mod_suphp behavior of executing scripts as the user specified by the domain portion of a URL.
paranoid_uid_check – Mod_SuPHP can be compiled with different security modes. EasyApache 3 uses the “paranoid” mode, though some scenarios are better served by the “force” mode. The paranoid UID check performed by mod_suphp verifies that the user ID that owns a script is the same one that is executing it. Setting this to false disables the UID check as in force mode.
paranoid_gid_check – Same as above, but with the group ID that owns the script. On some BSD systems it may be necessary to disable this check because files inherit the GID that owns the directory. Disabling this check should not significantly reduce security so long as the allow_file_group_writable and allow_directory_group_writable checks are left at their default values.
umask – The mod_suphp developers set this to a default of 0777 meaning that all permissions on files must be explicitly specified. This is a very secure setting, but causes many problems. The cPanel default is 0022. Use 0033, 0077, or 0777 for improved security.
min_uid – The lowest user ID that can own scripts being executed. Default value is 1. Set this value to 100 or 500 to greatly improve security. If you provide shared PHP scripts owned by root for your users, this value will need to be lowered to 0.
min_gid – The lowest group ID that can own scripts being executed. Default value is 1. Set this value to 100 or 500 to greatly improve security. If you provide shared PHP scripts owned by root for your users, this value will need to be lowered to 0.
full_php_process_display – When set to “true” mod_suphp will execute PHP scripts in a way that displays both the PHP interpreter and the SCRIPT_FILENAME in ps output. Setting this to “false” will improve security by hiding the SCRIPT_FILENAME.
The suphp.conf file includes a section called [phprc_paths] that can be used to override the standard handling of php.ini. To lock a particular PHP handler to its default php.ini file, simply uncomment the appropriate line under [phprc_paths]. There is also a configuration directive supported by mod_suphp in httpd.conf and .htaccess files called suPHP_ConfigPath that sets the path to the php.ini file. To prevent the use of this directive in .htaccess files, remove “Options” from the Apache AllowOverride setting. Note that the [phprc_paths] set in suphp.conf take precedence over any suPHP_ConfigPath settings.
FCGI Considerations
This is not a recommended configuration for PHP. It requires fine tuning of mod_fcgid to ensure that the server does not become overloaded with idle PHP processes.
Permissions of 0400 are sufficient to execute PHP scripts when running under Suexec. Permissions of 0444 are sufficient to execute PHP scripts when running with Suexec disabled.
With the standard cPanel FCGI configurations, the PHP binary is available as a URL in the VirtualHost. This should not be considered a very secure setup.
Userdir requests do not function with the FCGI setup provided by cPanel.
Apache directives like php_value are not valid for mod_fcgid. A custom php.ini file should be used instead as detailed in the SuPHP section above.
CGI Considerations
Like FCGI, this is not a recommended configuration for PHP. The PHP binary is available as a URL in the VirtualHost, and the setup is not very secure.
Permissions of 0400 are sufficient to execute PHP scripts when running under Suexec. Permissions of 0444 are sufficient to execute PHP scripts when running with Suexec disabled.
Several PHP options may prevent the CGI setup from functioning correctly, particularly DiscardPath and ForceCGIRedirect. If you are having trouble with this configuration, please verify these options are disabled.
Userdir requests do not function with the CGI setup provided by cPanel.
Apache directives like php_value are not valid for mod_cgi. A custom php.ini file should be used instead as detailed in the SuPHP section above.
Monday 9 July 2012
Apache configuration #2 - CentOS
Continuing from the first CentOS Apache configuration article, we now look at some of the other settings in the main httpd.conf file and what they can do.
Concentrating on efficiency and security, this will end our httpd.conf journey (for now).
Default: Not Set
The ServerName is usually a hostname or a FQDN (Fully Qualified Domain Name).
If you set followed the CentOS installing Apache and PHP5 article, you will have already set the ServerName configuration.
If you fail to set the ServerName then on an Apache restart you will see the following warning:
Default:
If you want happy users and to save traffic, keep this at Off.
Setting this to 'On' will enable DNS lookups so host names can be logged (it performs a reverse DNS check), setting it to 'Double' will not only perform the reverse DNS check it will then check the resulting hostname.
All a bit much and if you desperately need hostname information from your visitors it is advised to use logresolve (located in /usr/bin/logresolve) for this purpose. A small explanation can be found here.
Default:
The ServerTokens setting will dictate how much information is sent in the Headers with regard to Apache version and modules in use.
The default (Set as 'OS") would send something like this:
Does this make a difference? Well, yes. If we can suppress that information it will make it harder for someone to find an exploit.
It does not make the actual install any more secure but all someone has to do right now is look for an exploit in CentOS Apache 2.2.3 and so on. Why make it easy for them?
The options are (with example outputs):
Full
OS
Minimal
Minor
Major
Prod
It's up to you what level of info you want to give out. I prefer setting ServerTokens to Prod.
Default:
Server generated pages, such as 404 pages or directory listings, can contain a footer line which includes server information and can include the ServerAdmin email address.
If you navigate to your Slice IP address and a non-existent page:
You will see a 404 Page not found page with the footer information:
Note the image shown has ServerTokens set to 'Prod' so little information is shown in the footer.
The options are:
Off: Produces no footer
On: Produces footer information (at a level defined by the ServerTokens setting)
Email: Adds an email link to the information (level defined by the ServerTokens setting)
After each change to the httpd.conf file, you will need to reload Apache for the settings to take effect:
Some simple steps in this article but ones which I believe are very useful and aid in increasing the efficiency of your Slice and assist in the overall security effort on your Slice.
Concentrating on efficiency and security, this will end our httpd.conf journey (for now).
ServerName
Default: Not Set
The ServerName is usually a hostname or a FQDN (Fully Qualified Domain Name).
If you set followed the CentOS installing Apache and PHP5 article, you will have already set the ServerName configuration.
If you fail to set the ServerName then on an Apache restart you will see the following warning:
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name,
using 127.0.0.1 for ServerName
HostnameLookups
Default:
HostnameLookups Off
If you want happy users and to save traffic, keep this at Off.
Setting this to 'On' will enable DNS lookups so host names can be logged (it performs a reverse DNS check), setting it to 'Double' will not only perform the reverse DNS check it will then check the resulting hostname.
All a bit much and if you desperately need hostname information from your visitors it is advised to use logresolve (located in /usr/bin/logresolve) for this purpose. A small explanation can be found here.
ServerTokens
Default:
ServerTokens OS
The ServerTokens setting will dictate how much information is sent in the Headers with regard to Apache version and modules in use.
The default (Set as 'OS") would send something like this:
Apache/2.2.3 (CentOS)
Does this make a difference? Well, yes. If we can suppress that information it will make it harder for someone to find an exploit.
It does not make the actual install any more secure but all someone has to do right now is look for an exploit in CentOS Apache 2.2.3 and so on. Why make it easy for them?
The options are (with example outputs):
Full
Apache/2.2.3 (CentOS) DAV/2 PHP/5.1.6 mod_ssl/2.2.3 OpenSSL/0.9.8b
OS
Apache/2.2.3 (CentOS)
Minimal
Apache/2.2.3
Minor
Apache/2.2
Major
Apache/2
Prod
Apache
It's up to you what level of info you want to give out. I prefer setting ServerTokens to Prod.
ServerSignature
Default:
ServerSignature On
Server generated pages, such as 404 pages or directory listings, can contain a footer line which includes server information and can include the ServerAdmin email address.
If you navigate to your Slice IP address and a non-existent page:
http://123.45.67.890/blahblah
You will see a 404 Page not found page with the footer information:
Note the image shown has ServerTokens set to 'Prod' so little information is shown in the footer.
The options are:
Off: Produces no footer
On: Produces footer information (at a level defined by the ServerTokens setting)
Email: Adds an email link to the information (level defined by the ServerTokens setting)
Reload
After each change to the httpd.conf file, you will need to reload Apache for the settings to take effect:
sudo /etc/init.d/httpd reload
Summary
Some simple steps in this article but ones which I believe are very useful and aid in increasing the efficiency of your Slice and assist in the overall security effort on your Slice.
Apache configuration #1 - CentOS
Let's take a look at the main httpd.conf for our CentOS Apache install.
We're not actually going to change a lot at this point, just look at the main settings and see what they mean and what a change will actually do
Why no specific changes to the default? Well, it's difficult to give a definitive configuration as there are so many variables to consider such as expected site traffic, Slice size, site type, etc.
However, we'll discuss the main settings and you can make any decisions as to what settings you feel are best for your site.
My advice is very simple: experiment. Find what works best on your setup.
Open up the main CentOS Apache config file:
I won't list the whole contents here but, if you are not familiar with the settings, have a read of the comments. I find them very informative and straight to the point.
Let's look at some of the main settings and what they mean (you may notice that we skip some settings but don't worry, many of them will be discussed in the 2nd Apache configuration article):
Default:
This sets (in simple terms) the maximum time, in seconds, to wait for a request, action it and the response to the request.
The default is deliberately set high to allow for varied situations. You can reduce this to something more sane, such as 45 or even lower. A decrease may also help in reducing the effects of a DOS attack.
Default:
Setting this to 'On' allows for persistent connections to a client so each file, image, etc is not requested with a new connection. This allows for more efficiency. Define the KeepAlive settings as shown below:
Default:
Now we have our persistent connection, set the maximum number of requests per connection. Keep this high more maximum efficiency. If you have a site with images, javascripts, etc, try increasing this to 200.
Default:
So how long does the persistent connection wait for the next request? The default setting is very high and can easily be reduced to 2 or 3 seconds. If no new requests are received during this time the connection is killed.
What does this mean? Well, once a connection has been established and the client has requested the files needed for the web page, this setting says "sit there and ignore everyone else until the time limit is reached or you get a new request from the client".
Why would you want a higher time? In cases where there will be a lot of interactivity on the site. However, in most cases, people will go to a page, read it for a while and then click for the next page. You don't want the connection sat there doing nothing and ignoring other users.
During the Apache install we installed Apache using prefork and not Apache using worker. If you want to know more about the differences between the two I will point you towards the official Apache docs (which are actually very good).
Default:
Again, it's difficult to give a suggestion here as to what is best for your site but, most the time, they can be left at the defaults.
StartServers: number of child server processes created at startup
MinSpareServers: minimum number of child server processes not doing anything (idle).
MaxSpareServers: maximum number of child server processes not doing anything (idle) - any more than the maximum will be killed.
Don't set Max lower than Min but Apache will ignore silly numbers here and set the Max at Min+1.
ServerLimit: sets the server limit
MaxClients: sets the maximum simultaneous requests that Apache will handle. Anything over this number will be queued until a process is free to action the request.
MaxClients is not the same as the maximum number of visitors you can have. It is the maximum requests.
Remember the KeepAliveTimeout? This was set low so the next request can be actioned but the original (now 'idle') client will still be sat there reading your webpage - the new (active) request will be actioned or, if the MaxClients limit has been reached, will be queued ready for the next available process.
In most cases, the client is not 'active'. Take this page. You requested it (using an active process) and then spent a while reading it which uses no processes - you are 'idle' (as far as the server is concerned!).
MaxRequestsPerChild: sets how many requests a child process will handle before terminating. The default is 4000. If you set it to 0, it will never die.
Quite a lot here but as you go through the different settings you will see that the theory is quite simple. Naturally, there is a lot more to it than this article (or set of articles) can go into.
In the second httpd.conf article we will look at other settings that will add some more efficiency and help in increasing the security of our Slice.
We're not actually going to change a lot at this point, just look at the main settings and see what they mean and what a change will actually do
Defaults
Why no specific changes to the default? Well, it's difficult to give a definitive configuration as there are so many variables to consider such as expected site traffic, Slice size, site type, etc.
However, we'll discuss the main settings and you can make any decisions as to what settings you feel are best for your site.
My advice is very simple: experiment. Find what works best on your setup.
apache2.conf
Open up the main CentOS Apache config file:
sudo nano /etc/httpd/conf/httpd.conf
I won't list the whole contents here but, if you are not familiar with the settings, have a read of the comments. I find them very informative and straight to the point.
Let's look at some of the main settings and what they mean (you may notice that we skip some settings but don't worry, many of them will be discussed in the 2nd Apache configuration article):
Timeout
Default:
Timeout 120
This sets (in simple terms) the maximum time, in seconds, to wait for a request, action it and the response to the request.
The default is deliberately set high to allow for varied situations. You can reduce this to something more sane, such as 45 or even lower. A decrease may also help in reducing the effects of a DOS attack.
KeepAlive
Default:
KeepAlive Off
Setting this to 'On' allows for persistent connections to a client so each file, image, etc is not requested with a new connection. This allows for more efficiency. Define the KeepAlive settings as shown below:
MaxKeepAliveRequests
Default:
MaxKeepAliveRequests 100
Now we have our persistent connection, set the maximum number of requests per connection. Keep this high more maximum efficiency. If you have a site with images, javascripts, etc, try increasing this to 200.
KeepAliveTimeout
Default:
KeepAliveTimeout 15
So how long does the persistent connection wait for the next request? The default setting is very high and can easily be reduced to 2 or 3 seconds. If no new requests are received during this time the connection is killed.
What does this mean? Well, once a connection has been established and the client has requested the files needed for the web page, this setting says "sit there and ignore everyone else until the time limit is reached or you get a new request from the client".
Why would you want a higher time? In cases where there will be a lot of interactivity on the site. However, in most cases, people will go to a page, read it for a while and then click for the next page. You don't want the connection sat there doing nothing and ignoring other users.
prefork MPM
During the Apache install we installed Apache using prefork and not Apache using worker. If you want to know more about the differences between the two I will point you towards the official Apache docs (which are actually very good).
Default:
<IfModule mpm_prefork_module> StartServers 8 MinSpareServers 5 MaxSpareServers 20 ServerLimit 256 MaxClients 256 MaxRequestsPerChild 4000 </IfModule>
Again, it's difficult to give a suggestion here as to what is best for your site but, most the time, they can be left at the defaults.
StartServers: number of child server processes created at startup
MinSpareServers: minimum number of child server processes not doing anything (idle).
MaxSpareServers: maximum number of child server processes not doing anything (idle) - any more than the maximum will be killed.
Don't set Max lower than Min but Apache will ignore silly numbers here and set the Max at Min+1.
ServerLimit: sets the server limit
MaxClients: sets the maximum simultaneous requests that Apache will handle. Anything over this number will be queued until a process is free to action the request.
MaxClients is not the same as the maximum number of visitors you can have. It is the maximum requests.
Remember the KeepAliveTimeout? This was set low so the next request can be actioned but the original (now 'idle') client will still be sat there reading your webpage - the new (active) request will be actioned or, if the MaxClients limit has been reached, will be queued ready for the next available process.
In most cases, the client is not 'active'. Take this page. You requested it (using an active process) and then spent a while reading it which uses no processes - you are 'idle' (as far as the server is concerned!).
MaxRequestsPerChild: sets how many requests a child process will handle before terminating. The default is 4000. If you set it to 0, it will never die.
Summary
Quite a lot here but as you go through the different settings you will see that the theory is quite simple. Naturally, there is a lot more to it than this article (or set of articles) can go into.
In the second httpd.conf article we will look at other settings that will add some more efficiency and help in increasing the security of our Slice.
Script to monitor exim mail queue
==> Features
* This particular script alerts the admin if the mail queue is larger than the specified limit
* Script for EXIM Mail Servers
==> Usage
* Copy the below code to a file and give it execute permission
Code:
$ chmod 755 [file_name]
* Set the code to run every 30 minutes by configuring it as a cron job
Code:
*/30 * * * * /bin/sh [/path/to/file] > /dev/null 2>&1
* Edit the variables under the section: "Edit here" to suit your requirements
==> Script
Code:
#!/bin/bash
# Script from www.r6host.com to alert admin about larger mail queue
# Save the file as eximqueue.sh
######### Edit here ##########
_mail_user=info@r6host.com # Set this to your email id to receive alerts on mail queue
_limit=200 # Set the limit here
##############################
clear;
_result="/tmp/eximqueue.txt"
_queue="`exim-bpc`"
if [ "$_queue" -ge "$_limit" ]; then
echo "Current queue is: $_queue" > $_result
echo "Summary of Mail queue" >> $_result
echo "`exim -bp | exiqsumm`" >> $_result
mail -s "Number of mails on `hostname` : $_queue" $_mail_user < $_result
cat $_result
fi
rm -f $_result
* This particular script alerts the admin if the mail queue is larger than the specified limit
* Script for EXIM Mail Servers
==> Usage
* Copy the below code to a file and give it execute permission
Code:
$ chmod 755 [file_name]
* Set the code to run every 30 minutes by configuring it as a cron job
Code:
*/30 * * * * /bin/sh [/path/to/file] > /dev/null 2>&1
* Edit the variables under the section: "Edit here" to suit your requirements
==> Script
Code:
#!/bin/bash
# Script from www.r6host.com to alert admin about larger mail queue
# Save the file as eximqueue.sh
######### Edit here ##########
_mail_user=info@r6host.com # Set this to your email id to receive alerts on mail queue
_limit=200 # Set the limit here
##############################
clear;
_result="/tmp/eximqueue.txt"
_queue="`exim-bpc`"
if [ "$_queue" -ge "$_limit" ]; then
echo "Current queue is: $_queue" > $_result
echo "Summary of Mail queue" >> $_result
echo "`exim -bp | exiqsumm`" >> $_result
mail -s "Number of mails on `hostname` : $_queue" $_mail_user < $_result
cat $_result
fi
rm -f $_result
Sunday 8 July 2012
Counting Ips in an access log
So, If you need to look through an apache access log and count how many IP addresses there are this is how:
So this command will write out your access file to screen, use awk to print the first column, sort it, get the unique values, then sort it with a count, and print to "less".
Very useful to see if your being scraped!
root@sertechs [~]# cd /usr/local/apache/logs/
root@sertechs [~]# cat access_log | awk '{print$1}' | sort | uniq -c | sort -nr | less
So this command will write out your access file to screen, use awk to print the first column, sort it, get the unique values, then sort it with a count, and print to "less".
Very useful to see if your being scraped!
FIND Command
Some examples:
Find all files with 777 permissions:
find . -type f -perm 777 -print
Find all files that are older than 14 days and ls:
find . -type f -mtime +14 -exec ls -l {} ;
Find all your writable files and list them:
find . -perm -0777 -type f -ls
Find files and change their permission to writeable:
find . -name "NAME*" -exec chmod 666 {} ;
Find files created in the past 7 days:
find . -mtime -7 -type f
Find files modified in the past 7 days:
find . -mtime -7 -type f
Find files owned by a particular user:
find . -user esofthub
Find files that were modified more than 7 days ago and tar them into an archive (with a date in the name):
find . -type f -mtime +7 | xargs tar -cvf `date '+%d%m%Y'_archive.tar`
Find files that were modified less than 7 days ago and tar them into an archive (with a date in the name):
find . -type f -mtime -7 | xargs tar -cvf `date '+%d%m%Y'_archive.tar`
Find how many files are in a path:
find . -type f -exec basename {} ; | wc -l
Find the top 50 Biggest Files:
find / -type f -not -path "/proc/*" -not -path "/dev/*" -not -path "/sys/*" -not -path "/home/rack/*" -printf "%s %h/%fn" | sort -rn -k1 | head -n 50 | awk '{print $1/1048576 "MB" " " $2}'
Find all files with 777 permissions:
find . -type f -perm 777 -print
Find all files that are older than 14 days and ls:
find . -type f -mtime +14 -exec ls -l {} ;
Find all your writable files and list them:
find . -perm -0777 -type f -ls
Find files and change their permission to writeable:
find . -name "NAME*" -exec chmod 666 {} ;
Find files created in the past 7 days:
find . -mtime -7 -type f
Find files modified in the past 7 days:
find . -mtime -7 -type f
Find files owned by a particular user:
find . -user esofthub
Find files that were modified more than 7 days ago and tar them into an archive (with a date in the name):
find . -type f -mtime +7 | xargs tar -cvf `date '+%d%m%Y'_archive.tar`
Find files that were modified less than 7 days ago and tar them into an archive (with a date in the name):
find . -type f -mtime -7 | xargs tar -cvf `date '+%d%m%Y'_archive.tar`
Find how many files are in a path:
find . -type f -exec basename {} ; | wc -l
Find the top 50 Biggest Files:
find / -type f -not -path "/proc/*" -not -path "/dev/*" -not -path "/sys/*" -not -path "/home/rack/*" -printf "%s %h/%fn" | sort -rn -k1 | head -n 50 | awk '{print $1/1048576 "MB" " " $2}'
Install zend optimiser on direct admin server
cd /usr/local/directadmin/custombuild
./build update
vi /usr/local/directadmin/custombuild/options.conf
Change no to yes in options.conf
./build zend
./build update
vi /usr/local/directadmin/custombuild/options.conf
Change no to yes in options.conf
./build zend
How to enable open_basedir for particular domain in plesk
SSH into your Plesk machine as “root” and cd to the conf directory for the domain.
Create a “vhost.conf” file in “/var/www/vhosts/<domain name>/conf/”
<Directory /var/www/vhosts/<domain name>/httpdocs>
php_admin_value safe_mode on
php_admin_value open_basedir none
</Directory>
Rebuild the domain configs for the particular host via::–
[ #/usr/local/psa/admin/sbin/websrvmng -u --vhost-name=<domain name> ]
or rebuild all via :
#/usr/local/psa/admin/sbin/websrvmng -a
#service httpd reload
Create a “vhost.conf” file in “/var/www/vhosts/<domain name>/conf/”
<Directory /var/www/vhosts/<domain name>/httpdocs>
php_admin_value safe_mode on
php_admin_value open_basedir none
</Directory>
Rebuild the domain configs for the particular host via::–
[ #/usr/local/psa/admin/sbin/websrvmng -u --vhost-name=<domain name> ]
or rebuild all via :
#/usr/local/psa/admin/sbin/websrvmng -a
#service httpd reload
phpmyadmin upload limitation
To increase the phpmyadmin upload limitation in cpanel servers you can edit the php.ini file mentioned below :
vi /usr/local/cpanel/3rdparty/etc/phpmyadmin/php.ini.
upload_max_filesize = 100M
post_max_size = 100M
Even if you edit the main php.ini file it wont work in this case.
Cpanel >> phpmyadmin may not reflect the change, if so go to
WHM >> tweak settings >> cPanel PHP max POST size >> and increase the limit.
or
Increase WHM >> Tweak Settings >> cPanel PHP max upload size
vi /usr/local/cpanel/3rdparty/etc/phpmyadmin/php.ini.
upload_max_filesize = 100M
post_max_size = 100M
Even if you edit the main php.ini file it wont work in this case.
Cpanel >> phpmyadmin may not reflect the change, if so go to
WHM >> tweak settings >> cPanel PHP max POST size >> and increase the limit.
or
Increase WHM >> Tweak Settings >> cPanel PHP max upload size
How to disable CSF IP address block alert email
vi /etc/csf/csf.conf
# Send an email alert if an IP address is blocked by one of the [*] triggersLF_EMAIL_ALERT = “1?
# Send an email alert if an IP address is blocked by one of the [*] triggers
LF_EMAIL_ALERT = “1?
# Send an email alert if an IP address is blocked by one of the [*] triggersLF_EMAIL_ALERT = “1?
Thursday 5 July 2012
How to detect domain being Attacked or Attacking Out in cPanel
What we can do to find out which domain being attacked or attacking out from/to the server. Its no matter how this could happen, we need to stop that from happenning and turn our server stable. Its better to do this process in real-time within the time frame of server being attacked or the server others to make sure we can gather enough information, prove and logs. Its also recommended to document your process of troubleshooting for your reference. Believe me you will need it in future.
As for me, I will do basic checking as below:
1. Check overall server load summary using top command:
# top –c
2. Using the same command, we can monitor which process has taken high resource usage by sorting memory (Shift+M) or sorting CPU usage (Shift+P)
3. Check the network and analyse which connection flooding your server. Following command might be useful:
3.1 Check and sort number of network statistics connected to the server:
# netstat –anp |grep ‘tcp|udp’ | awk ‘{print $5}’ | cut –d: –f1 | sort | uniq –c | sort –n
3.2 If you have APFinstalled and using kernel older than 2.6.20, you can check the connection tracking table:
# cat /proc/net/ip_conntrack | cut –d ’ ’ –f 10 | cut –d ‘=’ –f 2 | sort | uniq –c | sort –nr | head –n 10
3.3 Do tcpdump to analyse packet that transmitted from/to your server. Following command might help to analyse any connection to eth0interface port 53 (DNS):
# tcpdump –vvxXlnni eth0 port 53 | grep A? | awk –F? ‘{print $2}’
4. Analyse Apache status page at WHM –> Server Status –> Apache Status. To do this via command line, you can run following command:
# service httpd fullstatus
5. Analyse Daily process logs at WHM –> Server Status –> Daily Process Logs. Find any top 5 users which consume most CPU percentage, memory and SQL process
After that, we should see some suspected account/process/user which occupied much resources either on CPU, memory or network connections.
Up until this part, we should shorlist any suspected account.
Then from the suspected account, we should do any step advised as below:
6. Scan the public_html directory of suspected user with anti virus. We can use clamav, but make sure the virus definition is updated before we do this:
6.1 Update clamavvirus definition:
# freshclam
6.2 Scan the public_html directory of the suspected user recursively with scan result logged to scanlog.txt:
# cd /home/user/public_html
# clamscan –i –r –l scanlog.txt &
6.3 Analyse any suspected files found by clamav and quarantine them. Make sure the file cannot be executed by chmod it to 600
7. Find any PHP files which contain suspicious characteristic like base64 encoded and store it into text file called scan_base64.txt.
Following command might help:
# cd /home/user/public_html
# grep –lir “eval(base64” *.php > scan_base64.txt
8. Scan the Apacheaccess log from raw log for any suspicious activities. Following command might help to find any scripting
activities happened in all domains via Apache:
# find /usr/local/apache/domlogs –exec egrep –iH ‘(wget|curl|lynx|gcc|perl|sh|cd|mkdir|touch)%20? {} ;
9. Analysing AWstats and bandwidth usage also get more clues. Go to cPanel > suspected domain > Logs > Awstats.
In the AWstats page, check the Hosts, Pages-URL or any related section. Example as below:
There are various way to help you in executing this task. As for me, above said steps should be enough to detect any domain/account
which attacking out or being attacked. Different administrator might using different approach in order to produce same result.
As for me, I will do basic checking as below:
1. Check overall server load summary using top command:
# top –c
2. Using the same command, we can monitor which process has taken high resource usage by sorting memory (Shift+M) or sorting CPU usage (Shift+P)
3. Check the network and analyse which connection flooding your server. Following command might be useful:
3.1 Check and sort number of network statistics connected to the server:
# netstat –anp |grep ‘tcp|udp’ | awk ‘{print $5}’ | cut –d: –f1 | sort | uniq –c | sort –n
3.2 If you have APFinstalled and using kernel older than 2.6.20, you can check the connection tracking table:
# cat /proc/net/ip_conntrack | cut –d ’ ’ –f 10 | cut –d ‘=’ –f 2 | sort | uniq –c | sort –nr | head –n 10
3.3 Do tcpdump to analyse packet that transmitted from/to your server. Following command might help to analyse any connection to eth0interface port 53 (DNS):
# tcpdump –vvxXlnni eth0 port 53 | grep A? | awk –F? ‘{print $2}’
4. Analyse Apache status page at WHM –> Server Status –> Apache Status. To do this via command line, you can run following command:
# service httpd fullstatus
5. Analyse Daily process logs at WHM –> Server Status –> Daily Process Logs. Find any top 5 users which consume most CPU percentage, memory and SQL process
After that, we should see some suspected account/process/user which occupied much resources either on CPU, memory or network connections.
Up until this part, we should shorlist any suspected account.
Then from the suspected account, we should do any step advised as below:
6. Scan the public_html directory of suspected user with anti virus. We can use clamav, but make sure the virus definition is updated before we do this:
6.1 Update clamavvirus definition:
# freshclam
6.2 Scan the public_html directory of the suspected user recursively with scan result logged to scanlog.txt:
# cd /home/user/public_html
# clamscan –i –r –l scanlog.txt &
6.3 Analyse any suspected files found by clamav and quarantine them. Make sure the file cannot be executed by chmod it to 600
7. Find any PHP files which contain suspicious characteristic like base64 encoded and store it into text file called scan_base64.txt.
Following command might help:
# cd /home/user/public_html
# grep –lir “eval(base64” *.php > scan_base64.txt
8. Scan the Apacheaccess log from raw log for any suspicious activities. Following command might help to find any scripting
activities happened in all domains via Apache:
# find /usr/local/apache/domlogs –exec egrep –iH ‘(wget|curl|lynx|gcc|perl|sh|cd|mkdir|touch)%20? {} ;
9. Analysing AWstats and bandwidth usage also get more clues. Go to cPanel > suspected domain > Logs > Awstats.
In the AWstats page, check the Hosts, Pages-URL or any related section. Example as below:
There are various way to help you in executing this task. As for me, above said steps should be enough to detect any domain/account
which attacking out or being attacked. Different administrator might using different approach in order to produce same result.
cPanel not showing FTP user accounts
This issue was due to the missing entries in the /etc/proftpd/username file.
For fixing this just issue the following command:-
# /scripts/ftpupdate.
This will sync the passwords of all the accounts in the servers
For fixing this just issue the following command:-
# /scripts/ftpupdate.
This will sync the passwords of all the accounts in the servers
Monday 2 July 2012
Php module installation steps :iconv
Overview :
Installing a single php extension without recompiling PHP is never been a difficult job but most of the people doesn’t know it which leads to re-compile whole php.. In this article i will explain how can you add new php extension without recompiling whole php.
In our example, i will tell you how can you add iconv php extension without recompiling PHP.
root@sysadmin [~]# php -m
To list all the php modules installed in the server
root@sysadmin[~]# php -m|grep iconv
To search for the php module iconv in the module list installed in the server
=============
root@sysadmin [~]# cd /home/cpeasyapache/src/php-5.2.9/ext/
root@sysadmin [/home/cpeasyapache/src/php-5.2.9/ext]# cd iconv/
root@sysadmin [/home/cpeasyapache/src/php-5.2.9/ext/iconv]# phpize
Configuring for:
PHP Api Version: 20041225
Zend Module Api No: 20060613
Zend Extension Api No: 220060519
===============
Phpize-- with an example
------
The phpize command is used to prepare the build environment for a PHP extension. In the following sample, the sources for an extension are in a directory named extname:
$ cd extname
$ phpize
$ ./configure
$ make
# make install
----------------------
You can can see iconv php extension is installed under php extensions directory:
Enable iconv PHP extension in php.ini
Verify iconv :
Output:
iconv support => enabled
Installing a single php extension without recompiling PHP is never been a difficult job but most of the people doesn’t know it which leads to re-compile whole php.. In this article i will explain how can you add new php extension without recompiling whole php.
In our example, i will tell you how can you add iconv php extension without recompiling PHP.
root@sysadmin [~]# php -m
To list all the php modules installed in the server
root@sysadmin[~]# php -m|grep iconv
To search for the php module iconv in the module list installed in the server
=============
root@sysadmin [~]# cd /home/cpeasyapache/src/php-5.2.9/ext/
root@sysadmin [/home/cpeasyapache/src/php-5.2.9/ext]# cd iconv/
root@sysadmin [/home/cpeasyapache/src/php-5.2.9/ext/iconv]# phpize
Configuring for:
PHP Api Version: 20041225
Zend Module Api No: 20060613
Zend Extension Api No: 220060519
===============
Phpize-- with an example
------
The phpize command is used to prepare the build environment for a PHP extension. In the following sample, the sources for an extension are in a directory named extname:
$ cd extname
$ phpize
$ ./configure
$ make
# make install
----------------------
You can can see iconv php extension is installed under php extensions directory:
ls /usr/local/lib/php/extensions/no-debug-non-zts-20060613/iconv.soEnable iconv PHP extension in php.ini
echo "extension=iconv.so" >> /usr/local/lib/php.iniVerify iconv :
php -i | grep -i "iconv support"Output:
iconv support => enabled
Saturday 16 June 2012
How to update stats in Plesk ?
Sometimes the webstat’s for a domain or for all domains on the server does not update automatically , in that case you can update the stat’s on the server easily by running the below commands :
To update the webstats for a particular domain :
root@server[#] /usr/local/psa/admin/sbin/statistics –calculate-one –domain-name=domainname
Note : Replace domain.com with the actual d
Thursday 31 May 2012
Shared IP Vs. Dedicated IP
Difference Between Shared IP and dedicated IP
All computers are connected to the Internet is assigned a unique IP address for the purposes of communication. An IP address is a 32-bit numeric address usually expressed as 4 numbers from 0-255 separated by dots, for example 192.168.0.123. There are billions of addresses possible, however, the number is finite.
All computers are connected to the Internet is assigned a unique IP address for the purposes of communication. An IP address is a 32-bit numeric address usually expressed as 4 numbers from 0-255 separated by dots, for example 192.168.0.123. There are billions of addresses possible, however, the number is finite.
How to Monitor Linux Server
Here you will get all Good tricks and tips how to monitor Linux Server or how to manage Linux Server if its causing load, How to find Spammer, how to check http and mysql processes.
General Commands,
To check server load and which users are logged on the server with IP address you can fire this command
w
To check for the server load and watch for process
top
top –d2
top –c d2
Memory status
free –m
To see all processes running on the server
ps –aufx
With above commands you can which process is causing load on the server after that you can go with next steps.
If you see many processes of exim then you can check exim in more detail. shows the total no of email in qmail
exim –bpc
Print a listing of the messages in the queue
exim -bp
Following command will show path to the script being utilized to send mail
ps -C exim -fH eww
ps -C exim -fH eww | grep home
cd /var/spool/exim/input/
egrep "X-PHP-Script" * -R
Shows no of frozen emails
exim -bpr | grep frozen | wc -l
To remove FROZEN mails from the server
exim -bp | exiqgrep -i | xargs exim -Mrm
exiqgrep -z -i | xargs exim –Mrm
Check for spamming if anybody is using php script for sending mail through home
tail -f /var/log/exim_mainlog | grep home
If anyone is spamming from /tmp
tail -f /var/log/exim_mainlog | grep /tmp
To display the IP and no of tries done bu the IP to send mail but rejected by the server.
tail -3000 /var/log/exim_mainlog |grep ‘rejected RCPT’ |awk ‘{print$4}’|awk -F[ '{print $2} '|awk -F] ‘{print $1} ‘|sort | uniq -c | sort -k 1 -nr | head -n 5
Shows the connections from a certain ip to the SMTP server
netstat -plan|grep :25|awk {‘print $5?}|cut -d: -f 1|sort|uniq -c|sort -nk 1
To shows the domain name and the no of emails sent by that domain
exim -bp | exiqsumm | more
If spamming from outside domain then you can block that domain or email id on the server
pico /etc/antivirus.exim
Add the following lines:
if $header_from: contains “name@domain.com”
then
seen finish
endif
Catching spammer
Check mail stats
exim -bp | exiqsumm | more
Following command will show you the maximum no of email currently in the mail queue have from or to the email address in the mail queue with exact figure.
exim -bpr | grep “” | awk ‘{print $4}’|grep -v “” | sort | uniq -c | sort -n
That will show you the maximum no of email currently in the mail queue have for the domain or from the domain with number.
exim -bpr | grep “” | awk ‘{print $4}’|grep -v “” |awk -F “@” ‘{ print $2}’ | sort | uniq -c | sort -n
Check if any php script is causing the mass mailing with
cd /var/spool/exim/input
egrep “X-PHP-Script” * -R
Just cat the ID that you get and you will be able to check which script is here causing problem for you.
To Remove particular email account email
exim -bpr |grep “ragnarockradio.org”|awk {‘print $3?}|xargs exim -Mrm
If Mysql causing the load so you can use following commands to check it.
mysqladmin pr
mysqladmin -u root processlist
mysqladmin version
watch mysqladmin proc
If Apache causing the load so check using following commands.
netstat -ntu | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort –n
netstat -an |grep :80 |wc –l
netstat -n | grep :80 | wc -l;uptime ; netstat -n | wc –l
netstat –tupl
pidof httpd
history | netstat
lsof -p pid
If mysql is causing load so you can check it using following commands.
mysqladmin -u root processlist
mysqladmin version
watch mysqladmin proc
mysqladmin -u root processlist
Other Useful Commands
To check ipd of php
pidof php
lsof -p pid
netstat -an |grep :80 |wc –l
netstat -ntu | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort -n
netstat -na |grep :80 |sort
Use below mentioned command to get top memory consuming processes
ps aux | head -1;ps aux –no-headers| sort -rn +3 | head
Use below command to get top cpu consuming processes
ps aux | head -1;ps aux –no-headers | sort -rn +2 |more
You can check if any backup is going on, run the following commands
ps aux | grep pkg
ps aux | grep gzip
ps aux | grep backup
We can trace the user responsible for high web server resource usage by the folowing command
cat /etc/httpd/logs/access_log | grep mp3
cat /etc/httpd/logs/access_log | grep rar
cat /etc/httpd/logs/access_log | grep wav etc
cat /etc/httpd/logs/access_log | grep 408 can be used to check for DDOS attacks on the server.
cat /etc/httpd/logs/access_log | grep rar
General Commands,
To check server load and which users are logged on the server with IP address you can fire this command
w
To check for the server load and watch for process
top
top –d2
top –c d2
Memory status
free –m
To see all processes running on the server
ps –aufx
With above commands you can which process is causing load on the server after that you can go with next steps.
If you see many processes of exim then you can check exim in more detail. shows the total no of email in qmail
exim –bpc
Print a listing of the messages in the queue
exim -bp
Following command will show path to the script being utilized to send mail
ps -C exim -fH eww
ps -C exim -fH eww | grep home
cd /var/spool/exim/input/
egrep "X-PHP-Script" * -R
Shows no of frozen emails
exim -bpr | grep frozen | wc -l
To remove FROZEN mails from the server
exim -bp | exiqgrep -i | xargs exim -Mrm
exiqgrep -z -i | xargs exim –Mrm
Check for spamming if anybody is using php script for sending mail through home
tail -f /var/log/exim_mainlog | grep home
If anyone is spamming from /tmp
tail -f /var/log/exim_mainlog | grep /tmp
To display the IP and no of tries done bu the IP to send mail but rejected by the server.
tail -3000 /var/log/exim_mainlog |grep ‘rejected RCPT’ |awk ‘{print$4}’|awk -F[ '{print $2} '|awk -F] ‘{print $1} ‘|sort | uniq -c | sort -k 1 -nr | head -n 5
Shows the connections from a certain ip to the SMTP server
netstat -plan|grep :25|awk {‘print $5?}|cut -d: -f 1|sort|uniq -c|sort -nk 1
To shows the domain name and the no of emails sent by that domain
exim -bp | exiqsumm | more
If spamming from outside domain then you can block that domain or email id on the server
pico /etc/antivirus.exim
Add the following lines:
if $header_from: contains “name@domain.com”
then
seen finish
endif
Catching spammer
Check mail stats
exim -bp | exiqsumm | more
Following command will show you the maximum no of email currently in the mail queue have from or to the email address in the mail queue with exact figure.
exim -bpr | grep “” | awk ‘{print $4}’|grep -v “” | sort | uniq -c | sort -n
That will show you the maximum no of email currently in the mail queue have for the domain or from the domain with number.
exim -bpr | grep “” | awk ‘{print $4}’|grep -v “” |awk -F “@” ‘{ print $2}’ | sort | uniq -c | sort -n
Check if any php script is causing the mass mailing with
cd /var/spool/exim/input
egrep “X-PHP-Script” * -R
Just cat the ID that you get and you will be able to check which script is here causing problem for you.
To Remove particular email account email
exim -bpr |grep “ragnarockradio.org”|awk {‘print $3?}|xargs exim -Mrm
If Mysql causing the load so you can use following commands to check it.
mysqladmin pr
mysqladmin -u root processlist
mysqladmin version
watch mysqladmin proc
If Apache causing the load so check using following commands.
netstat -ntu | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort –n
netstat -an |grep :80 |wc –l
netstat -n | grep :80 | wc -l;uptime ; netstat -n | wc –l
netstat –tupl
pidof httpd
history | netstat
lsof -p pid
If mysql is causing load so you can check it using following commands.
mysqladmin -u root processlist
mysqladmin version
watch mysqladmin proc
mysqladmin -u root processlist
Other Useful Commands
To check ipd of php
pidof php
lsof -p pid
netstat -an |grep :80 |wc –l
netstat -ntu | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort -n
netstat -na |grep :80 |sort
Use below mentioned command to get top memory consuming processes
ps aux | head -1;ps aux –no-headers| sort -rn +3 | head
Use below command to get top cpu consuming processes
ps aux | head -1;ps aux –no-headers | sort -rn +2 |more
You can check if any backup is going on, run the following commands
ps aux | grep pkg
ps aux | grep gzip
ps aux | grep backup
We can trace the user responsible for high web server resource usage by the folowing command
cat /etc/httpd/logs/access_log | grep mp3
cat /etc/httpd/logs/access_log | grep rar
cat /etc/httpd/logs/access_log | grep wav etc
cat /etc/httpd/logs/access_log | grep 408 can be used to check for DDOS attacks on the server.
cat /etc/httpd/logs/access_log | grep rar
Monday 28 May 2012
Linux / Unix Command: python
NAME
python - an interpreted, interactive, object-oriented programming language
SYNOPSIS
python [ -d ] [ -E ] [ -h ] [ -i ] [ -O ]
[ -Q argument ] [ -S ] [ -t ] [ -u ] [ -U ]
[ -v ] [ -V ] [ -W argument ] [ -x ]
[ -c command | script | - ] [ arguments ]
DESCRIPTION
Python is an interpreted, interactive, object-oriented programming language that combines remarkable power with very clear syntax. For an introduction to programming in Python you are referred to the Python Tutorial. The Python Library Reference documents built-in and standard types, constants, functions and modules. Finally, the Python Reference Manual describes the syntax and semantics of the core language in (perhaps too) much detail. (These documents may be located via the INTERNET RESOURCES below; they may be installed on your system as well.)
Python's basic power can be extended with your own modules written in C or C++. On most systems such modules may be dynamically loaded. Python is also adaptable as an extension language for existing applications. See the internal documentation for hints.
Documentation for installed Python modules and packages can be viewed by running thepydoc program.
COMMAND LINE OPTIONS
- -c command
- Specify the command to execute (see next section). This terminates the option list (following options are passed as arguments to the command).
- -d
- Turn on parser debugging output (for wizards only, depending on compilation options).
- -E
- Ignore environment variables like PYTHONPATH and PYTHONHOME that modify the behavior of the interpreter.
- -h
- Prints the usage for the interpreter executable and exits.
- -i
- When a script is passed as first argument or the -c option is used, enter interactive mode after executing the script or the command. It does not read the $PYTHONSTARTUP file. This can be useful to inspect global variables or a stack trace when a script raises an exception.
- -O
- Turn on basic optimizations. This changes the filename extension for compiled (bytecode) files from .pyc to .pyo. Given twice, causes docstrings to be discarded.
- -Q argument
- Division control; see PEP 238. The argument must be one of "old" (the default, int/int and long/long return an int or long), "new" (new division semantics, i.e. int/int and long/long returns a float), "warn" (old division semantics with a warning for int/int and long/long), or "warnall" (old division semantics with a warning for all use of the division operator). For a use of "warnall", see the Tools/scripts/fixdiv.py script.
- -S
- Disable the import of the module site and the site-dependent manipulations of sys.paththat it entails.
- -t
- Issue a warning when a source file mixes tabs and spaces for indentation in a way that makes it depend on the worth of a tab expressed in spaces. Issue an error when the option is given twice.
- -u
- Force stdin, stdout and stderr to be totally unbuffered.
- -v
- Print a message each time a module is initialized, showing the place (filename or built-in module) from which it is loaded. When given twice, print a message for each file that is checked for when searching for a module. Also provides information on module cleanup at exit.
- -V
- Prints the Python version number of the executable and exits.
- -W argument
- Warning control. Python sometimes prints warning message to sys.stderr. A typical warning message has the following form: file:line: category: message. By default, each warning is printed once for each source line where it occurs. This option controls how often warnings are printed. Multiple -W options may be given; when a warning matches more than one option, the action for the last matching option is performed. Invalid -Woptions are ignored (a warning message is printed about invalid options when the first warning is issued). Warnings can also be controlled from within a Python program using the warnings module.
The simplest form of argument is one of the following action strings (or a unique abbreviation): ignore to ignore all warnings; default to explicitly request the default behavior (printing each warning once per source line); all to print a warning each time it occurs (this may generate many messages if a warning is triggered repeatedly for the same source line, such as inside a loop); module to print each warning only only the first time it occurs in each module; once to print each warning only the first time it occurs in the program; or error to raise an exception instead of printing a warning message.
The full form of argument is action:message:category:module:line. Here, action is as explained above but only applies to messages that match the remaining fields. Empty fields match all values; trailing empty fields may be omitted. The message field matches the start of the warning message printed; this match is case-insensitive. The categoryfield matches the warning category. This must be a class name; the match test whether the actual warning category of the message is a subclass of the specified warning category. The full class name must be given. The module field matches the (fully-qualified) module name; this match is case-sensitive. The line field matches the line number, where zero matches all line numbers and is thus equivalent to an omitted line number.
- -x
- Skip the first line of the source. This is intended for a DOS specific hack only. Warning: the line numbers in error messages will be off by one!
INTERPRETER INTERFACE
The interpreter interface resembles that of the UNIX shell: when called with standard input connected to a tty device, it prompts for commands and executes them until an EOF is read; when called with a file name argument or with a file as standard input, it reads and executes a script from that file; when called with -c command, it executes the Python statement(s) given as command. Here command may contain multiple statements separated by newlines. Leading whitespace is significant in Python statements! In non-interactive mode, the entire input is parsed befored it is executed.
If available, the script name and additional arguments thereafter are passed to the script in the Python variable sys.argv , which is a list of strings (you must first import sys to be able to access it). If no script name is given, sys.argv[0] is an empty string; if -c is used, sys.argv[0]contains the string '-c'. Note that options interpreted by the Python interpreter itself are not placed in sys.argv.
In interactive mode, the primary prompt is `>>>'; the second prompt (which appears when a command is not complete) is `...'. The prompts can be changed by assignment to sys.ps1 orsys.ps2. The interpreter quits when it reads an EOF at a prompt. When an unhandled exception occurs, a stack trace is printed and control returns to the primary prompt; in non-interactive mode, the interpreter exits after printing the stack trace. The interrupt signal raises the KeyboardInterrupt exception; other UNIX signals are not caught (except that SIGPIPE is sometimes ignored, in favor of the IOError exception). Error messages are written to stderr.
FILES AND DIRECTORIES
These are subject to difference depending on local installation conventions; ${prefix} and ${exec_prefix} are installation-dependent and should be interpreted as for GNU software; they may be the same. The default for both is /usr/local.
- ${exec_prefix}/bin/python
- Recommended location of the interpreter.
${prefix}/lib/python<version>
${exec_prefix}/lib/python<version>
- Recommended locations of the directories containing the standard modules.
${prefix}/include/python<version>
${exec_prefix}/include/python<version>
- Recommended locations of the directories containing the include files needed for developing Python extensions and embedding the interpreter.
- ~/.pythonrc.py
- User-specific initialization file loaded by the user module; not used by default or by most applications.
Sunday 27 May 2012
cmdline
This is a linux command line reference for common operations. |
| Command | Description | |
| • | apropos whatis | Show commands pertinent to string. See also threadsafe |
| • | man -t ascii | ps2pdf - > ascii.pdf | make a pdf of a manual page |
| which command | Show full path name of command | |
| time command | See how long a command takes | |
| • | time cat | Start stopwatch. Ctrl-d to stop. See also sw |
| dir navigation | ||
| • | cd - | Go to previous directory |
| • | cd | Go to $HOME directory |
| (cd dir && command) | Go to dir, execute command and return to current dir | |
| • | pushd . | Put current dir on stack so you can popd back to it |
| file searching | ||
| • | alias l='ls -l --color=auto' | quick dir listing |
| • | ls -lrt | List files by date. See also newest and find_mm_yyyy |
| • | ls /usr/bin | pr -T9 -W$COLUMNS | Print in 9 columns to width of terminal |
| find -name '*.[ch]' | xargs grep -E 'expr' | Search 'expr' in this dir and below. See also findrepo | |
| find -type f -print0 | xargs -r0 grep -F 'example' | Search all regular files for 'example' in this dir and below | |
| find -maxdepth 1 -type f | xargs grep -F 'example' | Search all regular files for 'example' in this dir | |
| find -maxdepth 1 -type d | while read dir; do echo $dir; echo cmd2; done | Process each item with multiple commands (in while loop) | |
| • | find -type f ! -perm -444 | Find files not readable by all (useful for web site) |
| • | find -type d ! -perm -111 | Find dirs not accessible by all (useful for web site) |
| • | locate -r 'file[^/]*.txt' | Search cached index for names. This re is like glob *file*.txt |
| • | look reference | Quickly search (sorted) dictionary for prefix |
| • | grep --color reference /usr/share/dict/words | Highlight occurances of regular expression in dictionary |
| archives and compression | ||
| gpg -c file | Encrypt file | |
| gpg file.gpg | Decrypt file | |
| tar -c dir/ | bzip2 > dir.tar.bz2 | Make compressed archive of dir/ | |
| bzip2 -dc dir.tar.bz2 | tar -x | Extract archive (use gzip instead of bzip2 for tar.gz files) | |
| tar -c dir/ | gzip | gpg -c | ssh user@remote 'dd of=dir.tar.gz.gpg' | Make encrypted archive of dir/ on remote machine | |
| find dir/ -name '*.txt' | tar -c --files-from=- | bzip2 > dir_txt.tar.bz2 | Make archive of subset of dir/ and below | |
| find dir/ -name '*.txt' | xargs cp -a --target-directory=dir_txt/ --parents | Make copy of subset of dir/ and below | |
| ( tar -c /dir/to/copy ) | ( cd /where/to/ && tar -x -p ) | Copy (with permissions) copy/ dir to /where/to/ dir | |
| ( cd /dir/to/copy && tar -c . ) | ( cd /where/to/ && tar -x -p ) | Copy (with permissions) contents of copy/ dir to /where/to/ | |
| ( tar -c /dir/to/copy ) | ssh -C user@remote 'cd /where/to/ && tar -x -p' | Copy (with permissions) copy/ dir to remote:/where/to/ dir | |
| dd bs=1M if=/dev/sda | gzip | ssh user@remote 'dd of=sda.gz' | Backup harddisk to remote machine | |
| rsync (Network efficient file copier: Use the --dry-run option for testing) | ||
| rsync -P rsync://rsync.server.com/path/to/file file | Only get diffs. Do multiple times for troublesome downloads | |
| rsync --bwlimit=1000 fromfile tofile | Locally copy with rate limit. It's like nice for I/O | |
| rsync -az -e ssh --delete ~/public_html/ remote.com:'~/public_html' | Mirror web site (using compression and encryption) | |
| rsync -auz -e ssh remote:/dir/ . && rsync -auz -e ssh . remote:/dir/ | Synchronize current directory with remote one | |
| ssh (Secure SHell) | ||
| ssh $USER@$HOST command | Run command on $HOST as $USER (default command=shell) | |
| • | ssh -f -Y $USER@$HOSTNAME xeyes | Run GUI command on $HOSTNAME as $USER |
| scp -p -r $USER@$HOST: file dir/ | Copy with permissions to $USER's home directory on $HOST | |
| scp -c arcfour $USER@$LANHOST: bigfile | Use faster crypto for local LAN. This might saturate GigE | |
| ssh -g -L 8080:localhost:80 root@$HOST | Forward connections to $HOSTNAME:8080 out to $HOST:80 | |
| ssh -R 1434:imap:143 root@$HOST | Forward connections from $HOST:1434 in to imap:143 | |
| ssh-copy-id $USER@$HOST | Install public key for $USER@$HOST for password-less log in | |
| wget (multi purpose download tool) | ||
| • | (cd dir/ && wget -nd -pHEKk http://www.pixelbeat.org/cmdline.html) | Store local browsable version of a page to the current dir |
| wget -c http://www.example.com/large.file | Continue downloading a partially downloaded file | |
| wget -r -nd -np -l1 -A '*.jpg' http://www.example.com/dir/ | Download a set of files to the current directory | |
| wget ftp://remote/file[1-9].iso/ | FTP supports globbing directly | |
| • | wget -q -O- http://www.pixelbeat.org/timeline.html | grep 'a href' | head | Process output directly |
| echo 'wget url' | at 01:00 | Download url at 1AM to current dir | |
| wget --limit-rate=20k url | Do a low priority download (limit to 20KB/s in this case) | |
| wget -nv --spider --force-html -i bookmarks.html | Check links in a file | |
| wget --mirror http://www.example.com/ | Efficiently update a local copy of a site (handy from cron) | |
| networking (Note ifconfig, route, mii-tool, nslookup commands are obsolete) | ||
| ethtool eth0 | Show status of ethernet interface eth0 | |
| ethtool --change eth0 autoneg off speed 100 duplex full | Manually set ethernet interface speed | |
| iwconfig eth1 | Show status of wireless interface eth1 | |
| iwconfig eth1 rate 1Mb/s fixed | Manually set wireless interface speed | |
| • | iwlist scan | List wireless networks in range |
| • | ip link show | List network interfaces |
| ip link set dev eth0 name wan | Rename interface eth0 to wan | |
| ip link set dev eth0 up | Bring interface eth0 up (or down) | |
| • | ip addr show | List addresses for interfaces |
| ip addr add 1.2.3.4/24 brd + dev eth0 | Add (or del) ip and mask (255.255.255.0) | |
| • | ip route show | List routing table |
| ip route add default via 1.2.3.254 | Set default gateway to 1.2.3.254 | |
| • | host pixelbeat.org | Lookup DNS ip address for name or vice versa |
| • | hostname -i | Lookup local ip address (equivalent to host `hostname`) |
| • | whois pixelbeat.org | Lookup whois info for hostname or ip address |
| • | netstat -tupl | List internet services on a system |
| • | netstat -tup | List active connections to/from system |
| windows networking (Note samba is the package that provides all this windows specific networking support) | ||
| • | smbtree | Find windows machines. See also findsmb |
| nmblookup -A 1.2.3.4 | Find the windows (netbios) name associated with ip address | |
| smbclient -L windows_box | List shares on windows machine or samba server | |
| mount -t smbfs -o fmask=666,guest //windows_box/share /mnt/share | Mount a windows share | |
| echo 'message' | smbclient -M windows_box | Send popup to windows machine (off by default in XP sp2) | |
| text manipulation (Note sed uses stdin and stdout. Newer versions support inplace editing with the -i option) | ||
| sed 's/string1/string2/g' | Replace string1 with string2 | |
| sed 's/(.*)1/12/g' | Modify anystring1 to anystring2 | |
| sed '/ *#/d; /^ *$/d' | Remove comments and blank lines | |
| sed ':a; /$/N; s/n//; ta' | Concatenate lines with trailing | |
| sed 's/[ t]*$//' | Remove trailing spaces from lines | |
| sed 's/([`"$])/1/g' | Escape shell metacharacters active within double quotes | |
| • | seq 10 | sed "s/^/ /; s/ *(.{7,})/1/" | Right align numbers |
| sed -n '1000{p;q}' | Print 1000th line | |
| sed -n '10,20p;20q' | Print lines 10 to 20 | |
| sed -n 's/.*<title>(.*)</title>.*/1/ip;T;q' | Extract title from HTML web page | |
| sed -i 42d ~/.ssh/known_hosts | Delete a particular line | |
| sort -t. -k1,1n -k2,2n -k3,3n -k4,4n | Sort IPV4 ip addresses | |
| • | echo 'Test' | tr '[:lower:]' '[:upper:]' | Case conversion |
| • | tr -dc '[:print:]' < /dev/urandom | Filter non printable characters |
| • | tr -s '[:blank:]' 't' </proc/diskstats | cut -f4 | cut fields separated by blanks |
| • | history | wc -l | Count lines |
| set operations (Note you can export LANG=C for speed. Also these assume no duplicate lines within a file) | ||
| sort file1 file2 | uniq | Union of unsorted files | |
| sort file1 file2 | uniq -d | Intersection of unsorted files | |
| sort file1 file1 file2 | uniq -u | Difference of unsorted files | |
| sort file1 file2 | uniq -u | Symmetric Difference of unsorted files | |
| join -t'' -a1 -a2 file1 file2 | Union of sorted files | |
| join -t'' file1 file2 | Intersection of sorted files | |
| join -t'' -v2 file1 file2 | Difference of sorted files | |
| join -t'' -v1 -v2 file1 file2 | Symmetric Difference of sorted files | |
| math | ||
| • | echo '(1 + sqrt(5))/2' | bc -l | Quick math (Calculate ?). See also bc |
| • | seq -f '4/%g' 1 2 99999 | paste -sd-+ | bc -l | Calculate ? the unix way |
| • | echo 'pad=20; min=64; (100*10^6)/((pad+min)*8)' | bc | More complex (int) e.g. This shows max FastE packet rate |
| • | echo 'pad=20; min=64; print (100E6)/((pad+min)*8)' | python | Python handles scientific notation |
| • | echo 'pad=20; plot [64:1518] (100*10**6)/((pad+x)*8)' | gnuplot -persist | Plot FastE packet rate vs packet size |
| • | echo 'obase=16; ibase=10; 64206' | bc | Base conversion (decimal to hexadecimal) |
| • | echo $((0x2dec)) | Base conversion (hex to dec) ((shell arithmetic expansion)) |
| • | units -t '100m/9.58s' 'miles/hour' | Unit conversion (metric to imperial) |
| • | units -t '500GB' 'GiB' | Unit conversion (SI to IEC prefixes) |
| • | units -t '1 googol' | Definition lookup |
| • | seq 100 | (tr 'n' +; echo 0) | bc | Add a column of numbers. See also add and funcpy |
| calendar | ||
| • | cal -3 | Display a calendar |
| • | cal 9 1752 | Display a calendar for a particular month year |
| • | date -d fri | What date is it this friday. See also day |
| • | [ $(date -d '12:00 +1 day' +%d) = '01' ] || exit | exit a script unless it's the last day of the month |
| • | date --date='25 Dec' +%A | What day does xmas fall on, this year |
| • | date --date='@2147483647' | Convert seconds since the epoch (1970-01-01 UTC) to date |
| • | TZ='America/Los_Angeles' date | What time is it on west coast of US (use tzselect to find TZ) |
| • | date --date='TZ="America/Los_Angeles" 09:00 next Fri' | What's the local time for 9AM next Friday on west coast US |
| locales | ||
| • | printf "%'dn" 1234 | Print number with thousands grouping appropriate to locale |
| • | BLOCK_SIZE='1 ls -l | Use locale thousands grouping in ls. See also l |
| • | echo "I live in `locale territory`" | Extract info from locale database |
| • | LANG=en_IE.utf8 locale int_prefix | Lookup locale info for specific country. See also ccodes |
| • | locale -kc $(locale | sed -n 's/(LC_.{4,})=.*/1/p') | less | List fields available in locale database |
| recode (Obsoletes iconv, dos2unix, unix2dos) | ||
| • | recode -l | less | Show available conversions (aliases on each line) |
| recode windows-1252.. file_to_change.txt | Windows "ansi" to local charset (auto does CRLF conversion) | |
| recode utf-8/CRLF.. file_to_change.txt | Windows utf8 to local charset | |
| recode iso-8859-15..utf8 file_to_change.txt | Latin9 (western europe) to utf8 | |
| recode ../b64 < file.txt > file.b64 | Base64 encode | |
| recode /qp.. < file.qp > file.txt | Quoted printable decode | |
| recode ..HTML < file.txt > file.html | Text to HTML | |
| • | recode -lf windows-1252 | grep euro | Lookup table of characters |
| • | echo -n 0x80 | recode latin-9/x1..dump | Show what a code represents in latin-9 charmap |
| • | echo -n 0x20AC | recode ucs-2/x2..latin-9/x | Show latin-9 encoding |
| • | echo -n 0x20AC | recode ucs-2/x2..utf-8/x | Show utf-8 encoding |
| CDs | ||
| gzip < /dev/cdrom > cdrom.iso.gz | Save copy of data cdrom | |
| mkisofs -V LABEL -r dir | gzip > cdrom.iso.gz | Create cdrom image from contents of dir | |
| mount -o loop cdrom.iso /mnt/dir | Mount the cdrom image at /mnt/dir (read only) | |
| cdrecord -v dev=/dev/cdrom blank=fast | Clear a CDRW | |
| gzip -dc cdrom.iso.gz | cdrecord -v dev=/dev/cdrom - | Burn cdrom image (use dev=ATAPI -scanbus to confirm dev) | |
| cdparanoia -B | Rip audio tracks from CD to wav files in current dir | |
| cdrecord -v dev=/dev/cdrom -audio -pad *.wav | Make audio CD from all wavs in current dir (see also cdrdao) | |
| oggenc --tracknum='track' track.cdda.wav -o 'track.ogg' | Make ogg file from wav file | |
| disk space (See also FSlint) | ||
| • | ls -lSr | Show files by size, biggest last |
| • | du -s * | sort -k1,1rn | head | Show top disk users in current dir. See also dutop |
| • | du -hs /home/* | sort -k1,1h | Sort paths by easy to interpret disk usage |
| • | df -h | Show free space on mounted filesystems |
| • | df -i | Show free inodes on mounted filesystems |
| • | fdisk -l | Show disks partitions sizes and types (run as root) |
| • | rpm -q -a --qf '%10{SIZE}t%{NAME}n' | sort -k1,1n | List all packages by installed size (Bytes) on rpm distros |
| • | dpkg-query -W -f='${Installed-Size;10}t${Package}n' | sort -k1,1n | List all packages by installed size (KBytes) on deb distros |
| • | dd bs=1 seek=2TB if=/dev/null of=ext3.test | Create a large test file (taking no space). See also truncate |
| • | > file | truncate data of file or create an empty file |
| monitoring/debugging | ||
| • | tail -f /var/log/messages | Monitor messages in a log file |
| • | strace -c ls >/dev/null | Summarise/profile system calls made by command |
| • | strace -f -e open ls >/dev/null | List system calls made by command |
| • | strace -f -e trace=write -e write=1,2 ls >/dev/null | Monitor what's written to stdout and stderr |
| • | ltrace -f -e getenv ls >/dev/null | List library calls made by command |
| • | lsof -p $$ | List paths that process id has open |
| • | lsof ~ | List processes that have specified path open |
| • | tcpdump not port 22 | Show network traffic except ssh. See also tcpdump_not_me |
| • | ps -e -o pid,args --forest | List processes in a hierarchy |
| • | ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu | sed '/^ 0.0 /d' | List processes by % cpu usage |
| • | ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS | List processes by mem (KB) usage. See also ps_mem.py |
| • | ps -C firefox-bin -L -o pid,tid,pcpu,state | List all threads for a particular process |
| • | ps -p 1,$$ -o etime= | List elapsed wall time for particular process IDs |
| • | last reboot | Show system reboot history |
| • | free -m | Show amount of (remaining) RAM (-m displays in MB) |
| • | watch -n.1 'cat /proc/interrupts' | Watch changeable data continuously |
| • | udevadm monitor | Monitor udev events to help configure rules |
| system information (see also sysinfo) ('#' means root access is required) | ||
| • | uname -a | Show kernel version and system architecture |
| • | head -n1 /etc/issue | Show name and version of distribution |
| • | cat /proc/partitions | Show all partitions registered on the system |
| • | grep MemTotal /proc/meminfo | Show RAM total seen by the system |
| • | grep "model name" /proc/cpuinfo | Show CPU(s) info |
| • | lspci -tv | Show PCI info |
| • | lsusb -tv | Show USB info |
| • | mount | column -t | List mounted filesystems on the system (and align output) |
| • | grep -F capacity: /proc/acpi/battery/BAT0/info | Show state of cells in laptop battery |
| # | dmidecode -q | less | Display SMBIOS/DMI information |
| # | smartctl -A /dev/sda | grep Power_On_Hours | How long has this disk (system) been powered on in total |
| # | hdparm -i /dev/sda | Show info about disk sda |
| # | hdparm -tT /dev/sda | Do a read speed test on disk sda |
| # | badblocks -s /dev/sda | Test for unreadable blocks on disk sda |
| interactive (see also linux keyboard shortcuts) | ||
| • | readline | Line editor used by bash, python, bc, gnuplot, ... |
| • | screen | Virtual terminals with detach capability, ... |
| • | mc | Powerful file manager that can browse rpm, tar, ftp, ssh, ... |
| • | gnuplot | Interactive/scriptable graphing |
| • | links | Web browser |
| • | xdg-open . | open a file or url with the registered desktop application |
More Linux Commands
Examples marked with • are valid/safe to paste without modification into a terminal, so
you may want to keep a terminal window open while reading this so you can cut & paste.
| Command | Description | |
| • | grep . /proc/sys/net/ipv4/* | List the contents of flag files |
| • | set | grep $USER | Search current environment |
| • | tr '' 'n' < /proc/$$/environ | Display the startup environment for any process |
| • | echo $PATH | tr : 'n' | Display the $PATH one per line |
| • | kill -0 $$ && echo process exists and can accept signals | Check for the existence of a process (pid) |
| • | find /etc -readable | xargs less -K -p'*ntp' -j $((${LINES:-25}/2)) | Search paths and data with full context. Use n to iterate |
| Low impact admin | ||
| # | apt-get install "package" -o Acquire::http::Dl-Limit=42 -o Acquire::Queue-mode=access | Rate limit apt-get to 42KB/s |
| echo 'wget url' | at 01:00 | Download url at 1AM to current dir | |
| # | apache2ctl configtest && apache2ctl graceful | Restart apache if config is OK |
| • | nice openssl speed sha1 | Run a low priority command (openssl benchmark) |
| • | chrt -i 0 openssl speed sha1 | Run a low priority command (more effective than nice) |
| • | renice 19 -p $$; ionice -c3 -p $$ | Make shell (script) low priority. Use for non interactive tasks |
| Interactive monitoring | ||
| • | watch -t -n1 uptime | Clock with system load |
| • | htop -d 5 | Better top (scrollable, tree view, lsof/strace integration, ...) |
| • | iotop | What's doing I/O |
| # | watch -d -n30 "nice ps_mem.py | tail -n $((${LINES:-12}-2))" | What's using RAM |
| # | iftop | What's using the network. See also iptraf |
| # | mtr www.pixelbeat.org | ping and traceroute combined |
| Useful utilities | ||
| • | pv < /dev/zero > /dev/null | Progress Viewer for data copying from files and pipes |
| • | wkhtml2pdf http://.../linux_commands.html linux_commands.pdf | Make a pdf of a web page |
| • | timeout 1 sleep 3 | run a command with bounded time. See also timeout |
| Networking | ||
| • | python -m SimpleHTTPServer | Serve current directory tree at http://$HOSTNAME:8000/ |
| • | openssl s_client -connect www.google.com:443 </dev/null 2>&0 | openssl x509 -dates -noout | Display the date range for a site's certs |
| • | curl -I www.pixelbeat.org | Display the server headers for a web site |
| # | lsof -i tcp:80 | What's using port 80 |
| # | httpd -S | Display a list of apache virtual hosts |
| • | vim scp://user@remote//path/to/file | Edit remote file using local vim. Good for high latency links |
| • | curl -s http://www.pixelbeat.org/pixelbeat.asc | gpg --import | Import a gpg key from the web |
| • | tc qdisc add dev lo root handle 1:0 netem delay 20msec | Add 20ms latency to loopback device (for testing) |
| • | tc qdisc del dev lo root | Remove latency added above |
| Notification | ||
| • | echo "DISPLAY=$DISPLAY xmessage cooker" | at "NOW +30min" | Popup reminder |
| • | notify-send "subject" "message" | Display a gnome popup notification |
| echo "mail -s 'go home' P@draigBrady.com < /dev/null" | at 17:30 | Email reminder | |
| uuencode file name | mail -s subject P@draigBrady.com | Send a file via email | |
| ansi2html.sh | mail -a "Content-Type: text/html" P@draigBrady.com | Send/Generate HTML email | |
| Better default settings (useful in your .bashrc) | ||
| # | tail -s.1 -f /var/log/messages | Display file additions more responsively |
| • | seq 100 | tail -n $((${LINES:-12}-2)) | Display as many lines as possible without scrolling |
| # | tcpdump -s0 | Capture full network packets |
| Useful functions/aliases (useful in your .bashrc) | ||
| • | md () { mkdir -p "$1" && cd "$1"; } | Change to a new directory |
| • | strerror() { python -c "import os; print os.strerror($1)"; } | Display the meaning of an errno |
| • | plot() { { echo 'plot "-"' "$@"; cat; } | gnuplot -persist; } | Plot stdin. (e.g: • seq 1000 | sed 's/.*/s(&)/' | bc -l | plot) |
| • | hili() { e="$1"; shift; grep --col=always -Eih "$e|$" "$@"; } | highlight occurences of expr. (e.g: • env | hili $USER) |
| • | alias hd='od -Ax -tx1z -v' | Hexdump. (usage e.g.: • hd /proc/self/cmdline | less) |
| • | alias realpath='readlink -f' | Canonicalize path. (usage e.g.: • realpath ~/../$USER) |
| • | ord() { printf "0x%xn" "'$1"; } | shell version of the ord() function |
| • | chr() { printf $(printf '%03on' "$1"); } | shell version of the chr() function |
| Multimedia | ||
| • | DISPLAY=:0.0 import -window root orig.png | Take a (remote) screenshot |
| • | convert -filter catrom -resize '600x>' orig.png 600px_wide.png | Shrink to width, computer gen images or screenshots |
| mplayer -ao pcm -vo null -vc dummy /tmp/Flash* | Extract audio from flash video to audiodump.wav | |
| ffmpeg -i filename.avi | Display info about multimedia file | |
| • | ffmpeg -f x11grab -s xga -r 25 -i :0 -sameq demo.mpg | Capture video of an X display |
| DVD | ||
| for i in $(seq 9); do ffmpeg -i $i.avi -target pal-dvd $i.mpg; done | Convert video to the correct encoding and aspect for DVD | |
| dvdauthor -odvd -t -v "pal,4:3,720xfull" *.mpg;dvdauthor -odvd -T | Build DVD file system. Use 16:9 for widescreen input | |
| growisofs -dvd-compat -Z /dev/dvd -dvd-video dvd | Burn DVD file system to disc | |
| Unicode | ||
| • | python -c "import unicodedata as u; print u.name(unichr(0x2028))" | Lookup a unicode character |
| • | uconv -f utf8 -t utf8 -x nfc | Normalize combining characters |
| • | printf '300200' | iconv -futf8 -tutf8 >/dev/null | Validate UTF-8 |
| • | printf '?TF8n' | LANG=C grep --color=always '[^ -~]+' | Highlight non printable ASCII chars in UTF-8 |
| • | fc-match -s "sans:lang=zh" | List font match order for language and style |
| Development | ||
| • | gcc -march=native -E -v -</dev/null 2>&1|sed -n 's/.*-mar/-mar/p' | Show autodetected gcc tuning params. See also gcccpuopt |
| • | for i in $(seq 4); do { [ $i = 1 ] && wget http://url.ie/6lko -qO-|| ./a.out; } | tee /dev/tty | gcc -xc - 2>/dev/null; done | Compile and execute C code from stdin |
| • | cpp -dM /dev/null | Show all predefined macros |
| • | echo "#include <features.h>" | cpp -dN | grep "#define __USE_" | Show all glibc feature macros |
| gdb -tui | Debug showing source code context in separate windows | |
| udev | ||
| • | udevadm info -a -p $(udevadm info -q path -n /dev/input/mouse0) | List udev attributes of a device, for matching rules etc. |
| • | udevadm test /sys/class/input/mouse0 | See how udev rules are applied for a device |
| # | udevadm control --reload-rules | Reload udev rules after modification |
| Extended Attributes (Note you may need to (re)mount with "acl" or "user_xattr" options) | ||
| • | getfacl . | Show ACLs for file |
| • | setfacl -m u:nobody:r . | Allow a specific user to read file |
| • | setfacl -x u:nobody . | Delete a specific user's rights to file |
| setfacl --default -m group:users:rw- dir/ | Set umask for a for a specific dir | |
| getcap file | Show capabilities for a program | |
| setcap cap_net_raw+ep your_gtk_prog | Allow gtk program raw access to network | |
| • | stat -c%C . | Show SELinux context for file |
| chcon ... file | Set SELinux context for file (see also restorecon) | |
| • | getfattr -m- -d . | Show all extended attributes (includes selinux,acls,...) |
| • | setfattr -n "user.foo" -v "bar" . | Set arbitrary user attributes |
| BASH specific | ||
| • | echo 123 | tee >(tr 1 a) | tr 1 b | Split data to 2 commands (using process substitution) |
| meld local_file <(ssh host cat remote_file) | Compare a local and remote file (using process substitution) | |
| Multicore | ||
| • | taskset -c 0 nproc | Restrict a command to certain processors |
| • | find -type f -print0 | xargs -r0 -P$(nproc) -n10 md5sum | Process files in parallel over available processors |
| sort -m <(sort data1) <(sort data2) >data.sorted | Sort separate data files over 2 processors | |
Subscribe to:
Posts (Atom)