Thursday, 31 May 2012
Shared IP Vs. Dedicated IP
All computers are connected to the Internet is assigned a unique IP address for the purposes of communication. An IP address is a 32-bit numeric address usually expressed as 4 numbers from 0-255 separated by dots, for example 192.168.0.123. There are billions of addresses possible, however, the number is finite.
How to Monitor Linux Server
General Commands,
To check server load and which users are logged on the server with IP address you can fire this command
w
To check for the server load and watch for process
top
top –d2
top –c d2
Memory status
free –m
To see all processes running on the server
ps –aufx
With above commands you can which process is causing load on the server after that you can go with next steps.
If you see many processes of exim then you can check exim in more detail. shows the total no of email in qmail
exim –bpc
Print a listing of the messages in the queue
exim -bp
Following command will show path to the script being utilized to send mail
ps -C exim -fH eww
ps -C exim -fH eww | grep home
cd /var/spool/exim/input/
egrep "X-PHP-Script" * -R
Shows no of frozen emails
exim -bpr | grep frozen | wc -l
To remove FROZEN mails from the server
exim -bp | exiqgrep -i | xargs exim -Mrm
exiqgrep -z -i | xargs exim –Mrm
Check for spamming if anybody is using php script for sending mail through home
tail -f /var/log/exim_mainlog | grep home
If anyone is spamming from /tmp
tail -f /var/log/exim_mainlog | grep /tmp
To display the IP and no of tries done bu the IP to send mail but rejected by the server.
tail -3000 /var/log/exim_mainlog |grep ‘rejected RCPT’ |awk ‘{print$4}’|awk -F[ '{print $2} '|awk -F] ‘{print $1} ‘|sort | uniq -c | sort -k 1 -nr | head -n 5
Shows the connections from a certain ip to the SMTP server
netstat -plan|grep :25|awk {‘print $5?}|cut -d: -f 1|sort|uniq -c|sort -nk 1
To shows the domain name and the no of emails sent by that domain
exim -bp | exiqsumm | more
If spamming from outside domain then you can block that domain or email id on the server
pico /etc/antivirus.exim
Add the following lines:
if $header_from: contains “name@domain.com”
then
seen finish
endif
Catching spammer
Check mail stats
exim -bp | exiqsumm | more
Following command will show you the maximum no of email currently in the mail queue have from or to the email address in the mail queue with exact figure.
exim -bpr | grep “” | awk ‘{print $4}’|grep -v “” | sort | uniq -c | sort -n
That will show you the maximum no of email currently in the mail queue have for the domain or from the domain with number.
exim -bpr | grep “” | awk ‘{print $4}’|grep -v “” |awk -F “@” ‘{ print $2}’ | sort | uniq -c | sort -n
Check if any php script is causing the mass mailing with
cd /var/spool/exim/input
egrep “X-PHP-Script” * -R
Just cat the ID that you get and you will be able to check which script is here causing problem for you.
To Remove particular email account email
exim -bpr |grep “ragnarockradio.org”|awk {‘print $3?}|xargs exim -Mrm
If Mysql causing the load so you can use following commands to check it.
mysqladmin pr
mysqladmin -u root processlist
mysqladmin version
watch mysqladmin proc
If Apache causing the load so check using following commands.
netstat -ntu | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort –n
netstat -an |grep :80 |wc –l
netstat -n | grep :80 | wc -l;uptime ; netstat -n | wc –l
netstat –tupl
pidof httpd
history | netstat
lsof -p pid
If mysql is causing load so you can check it using following commands.
mysqladmin -u root processlist
mysqladmin version
watch mysqladmin proc
mysqladmin -u root processlist
Other Useful Commands
To check ipd of php
pidof php
lsof -p pid
netstat -an |grep :80 |wc –l
netstat -ntu | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort -n
netstat -na |grep :80 |sort
Use below mentioned command to get top memory consuming processes
ps aux | head -1;ps aux –no-headers| sort -rn +3 | head
Use below command to get top cpu consuming processes
ps aux | head -1;ps aux –no-headers | sort -rn +2 |more
You can check if any backup is going on, run the following commands
ps aux | grep pkg
ps aux | grep gzip
ps aux | grep backup
We can trace the user responsible for high web server resource usage by the folowing command
cat /etc/httpd/logs/access_log | grep mp3
cat /etc/httpd/logs/access_log | grep rar
cat /etc/httpd/logs/access_log | grep wav etc
cat /etc/httpd/logs/access_log | grep 408 can be used to check for DDOS attacks on the server.
cat /etc/httpd/logs/access_log | grep rar
Monday, 28 May 2012
Linux / Unix Command: python
NAME
python - an interpreted, interactive, object-oriented programming language
SYNOPSIS
python [ -d ] [ -E ] [ -h ] [ -i ] [ -O ]
[ -Q argument ] [ -S ] [ -t ] [ -u ] [ -U ]
[ -v ] [ -V ] [ -W argument ] [ -x ]
[ -c command | script | - ] [ arguments ]
DESCRIPTION
Python is an interpreted, interactive, object-oriented programming language that combines remarkable power with very clear syntax. For an introduction to programming in Python you are referred to the Python Tutorial. The Python Library Reference documents built-in and standard types, constants, functions and modules. Finally, the Python Reference Manual describes the syntax and semantics of the core language in (perhaps too) much detail. (These documents may be located via the INTERNET RESOURCES below; they may be installed on your system as well.)
Python's basic power can be extended with your own modules written in C or C++. On most systems such modules may be dynamically loaded. Python is also adaptable as an extension language for existing applications. See the internal documentation for hints.
Documentation for installed Python modules and packages can be viewed by running thepydoc program.
COMMAND LINE OPTIONS
- -c command
- Specify the command to execute (see next section). This terminates the option list (following options are passed as arguments to the command).
- -d
- Turn on parser debugging output (for wizards only, depending on compilation options).
- -E
- Ignore environment variables like PYTHONPATH and PYTHONHOME that modify the behavior of the interpreter.
- -h
- Prints the usage for the interpreter executable and exits.
- -i
- When a script is passed as first argument or the -c option is used, enter interactive mode after executing the script or the command. It does not read the $PYTHONSTARTUP file. This can be useful to inspect global variables or a stack trace when a script raises an exception.
- -O
- Turn on basic optimizations. This changes the filename extension for compiled (bytecode) files from .pyc to .pyo. Given twice, causes docstrings to be discarded.
- -Q argument
- Division control; see PEP 238. The argument must be one of "old" (the default, int/int and long/long return an int or long), "new" (new division semantics, i.e. int/int and long/long returns a float), "warn" (old division semantics with a warning for int/int and long/long), or "warnall" (old division semantics with a warning for all use of the division operator). For a use of "warnall", see the Tools/scripts/fixdiv.py script.
- -S
- Disable the import of the module site and the site-dependent manipulations of sys.paththat it entails.
- -t
- Issue a warning when a source file mixes tabs and spaces for indentation in a way that makes it depend on the worth of a tab expressed in spaces. Issue an error when the option is given twice.
- -u
- Force stdin, stdout and stderr to be totally unbuffered.
- -v
- Print a message each time a module is initialized, showing the place (filename or built-in module) from which it is loaded. When given twice, print a message for each file that is checked for when searching for a module. Also provides information on module cleanup at exit.
- -V
- Prints the Python version number of the executable and exits.
- -W argument
- Warning control. Python sometimes prints warning message to sys.stderr. A typical warning message has the following form: file:line: category: message. By default, each warning is printed once for each source line where it occurs. This option controls how often warnings are printed. Multiple -W options may be given; when a warning matches more than one option, the action for the last matching option is performed. Invalid -Woptions are ignored (a warning message is printed about invalid options when the first warning is issued). Warnings can also be controlled from within a Python program using the warnings module.
The simplest form of argument is one of the following action strings (or a unique abbreviation): ignore to ignore all warnings; default to explicitly request the default behavior (printing each warning once per source line); all to print a warning each time it occurs (this may generate many messages if a warning is triggered repeatedly for the same source line, such as inside a loop); module to print each warning only only the first time it occurs in each module; once to print each warning only the first time it occurs in the program; or error to raise an exception instead of printing a warning message.
The full form of argument is action:message:category:module:line. Here, action is as explained above but only applies to messages that match the remaining fields. Empty fields match all values; trailing empty fields may be omitted. The message field matches the start of the warning message printed; this match is case-insensitive. The categoryfield matches the warning category. This must be a class name; the match test whether the actual warning category of the message is a subclass of the specified warning category. The full class name must be given. The module field matches the (fully-qualified) module name; this match is case-sensitive. The line field matches the line number, where zero matches all line numbers and is thus equivalent to an omitted line number.
- -x
- Skip the first line of the source. This is intended for a DOS specific hack only. Warning: the line numbers in error messages will be off by one!
INTERPRETER INTERFACE
The interpreter interface resembles that of the UNIX shell: when called with standard input connected to a tty device, it prompts for commands and executes them until an EOF is read; when called with a file name argument or with a file as standard input, it reads and executes a script from that file; when called with -c command, it executes the Python statement(s) given as command. Here command may contain multiple statements separated by newlines. Leading whitespace is significant in Python statements! In non-interactive mode, the entire input is parsed befored it is executed.
If available, the script name and additional arguments thereafter are passed to the script in the Python variable sys.argv , which is a list of strings (you must first import sys to be able to access it). If no script name is given, sys.argv[0] is an empty string; if -c is used, sys.argv[0]contains the string '-c'. Note that options interpreted by the Python interpreter itself are not placed in sys.argv.
In interactive mode, the primary prompt is `>>>'; the second prompt (which appears when a command is not complete) is `...'. The prompts can be changed by assignment to sys.ps1 orsys.ps2. The interpreter quits when it reads an EOF at a prompt. When an unhandled exception occurs, a stack trace is printed and control returns to the primary prompt; in non-interactive mode, the interpreter exits after printing the stack trace. The interrupt signal raises the KeyboardInterrupt exception; other UNIX signals are not caught (except that SIGPIPE is sometimes ignored, in favor of the IOError exception). Error messages are written to stderr.
FILES AND DIRECTORIES
These are subject to difference depending on local installation conventions; ${prefix} and ${exec_prefix} are installation-dependent and should be interpreted as for GNU software; they may be the same. The default for both is /usr/local.
- ${exec_prefix}/bin/python
- Recommended location of the interpreter.
${prefix}/lib/python<version>
${exec_prefix}/lib/python<version>
- Recommended locations of the directories containing the standard modules.
${prefix}/include/python<version>
${exec_prefix}/include/python<version>
- Recommended locations of the directories containing the include files needed for developing Python extensions and embedding the interpreter.
- ~/.pythonrc.py
- User-specific initialization file loaded by the user module; not used by default or by most applications.
Sunday, 27 May 2012
cmdline
This is a linux command line reference for common operations. |
Command | Description | |
• | apropos whatis | Show commands pertinent to string. See also threadsafe |
• | man -t ascii | ps2pdf - > ascii.pdf | make a pdf of a manual page |
which command | Show full path name of command | |
time command | See how long a command takes | |
• | time cat | Start stopwatch. Ctrl-d to stop. See also sw |
dir navigation | ||
• | cd - | Go to previous directory |
• | cd | Go to $HOME directory |
(cd dir && command) | Go to dir, execute command and return to current dir | |
• | pushd . | Put current dir on stack so you can popd back to it |
file searching | ||
• | alias l='ls -l --color=auto' | quick dir listing |
• | ls -lrt | List files by date. See also newest and find_mm_yyyy |
• | ls /usr/bin | pr -T9 -W$COLUMNS | Print in 9 columns to width of terminal |
find -name '*.[ch]' | xargs grep -E 'expr' | Search 'expr' in this dir and below. See also findrepo | |
find -type f -print0 | xargs -r0 grep -F 'example' | Search all regular files for 'example' in this dir and below | |
find -maxdepth 1 -type f | xargs grep -F 'example' | Search all regular files for 'example' in this dir | |
find -maxdepth 1 -type d | while read dir; do echo $dir; echo cmd2; done | Process each item with multiple commands (in while loop) | |
• | find -type f ! -perm -444 | Find files not readable by all (useful for web site) |
• | find -type d ! -perm -111 | Find dirs not accessible by all (useful for web site) |
• | locate -r 'file[^/]*.txt' | Search cached index for names. This re is like glob *file*.txt |
• | look reference | Quickly search (sorted) dictionary for prefix |
• | grep --color reference /usr/share/dict/words | Highlight occurances of regular expression in dictionary |
archives and compression | ||
gpg -c file | Encrypt file | |
gpg file.gpg | Decrypt file | |
tar -c dir/ | bzip2 > dir.tar.bz2 | Make compressed archive of dir/ | |
bzip2 -dc dir.tar.bz2 | tar -x | Extract archive (use gzip instead of bzip2 for tar.gz files) | |
tar -c dir/ | gzip | gpg -c | ssh user@remote 'dd of=dir.tar.gz.gpg' | Make encrypted archive of dir/ on remote machine | |
find dir/ -name '*.txt' | tar -c --files-from=- | bzip2 > dir_txt.tar.bz2 | Make archive of subset of dir/ and below | |
find dir/ -name '*.txt' | xargs cp -a --target-directory=dir_txt/ --parents | Make copy of subset of dir/ and below | |
( tar -c /dir/to/copy ) | ( cd /where/to/ && tar -x -p ) | Copy (with permissions) copy/ dir to /where/to/ dir | |
( cd /dir/to/copy && tar -c . ) | ( cd /where/to/ && tar -x -p ) | Copy (with permissions) contents of copy/ dir to /where/to/ | |
( tar -c /dir/to/copy ) | ssh -C user@remote 'cd /where/to/ && tar -x -p' | Copy (with permissions) copy/ dir to remote:/where/to/ dir | |
dd bs=1M if=/dev/sda | gzip | ssh user@remote 'dd of=sda.gz' | Backup harddisk to remote machine | |
rsync (Network efficient file copier: Use the --dry-run option for testing) | ||
rsync -P rsync://rsync.server.com/path/to/file file | Only get diffs. Do multiple times for troublesome downloads | |
rsync --bwlimit=1000 fromfile tofile | Locally copy with rate limit. It's like nice for I/O | |
rsync -az -e ssh --delete ~/public_html/ remote.com:'~/public_html' | Mirror web site (using compression and encryption) | |
rsync -auz -e ssh remote:/dir/ . && rsync -auz -e ssh . remote:/dir/ | Synchronize current directory with remote one | |
ssh (Secure SHell) | ||
ssh $USER@$HOST command | Run command on $HOST as $USER (default command=shell) | |
• | ssh -f -Y $USER@$HOSTNAME xeyes | Run GUI command on $HOSTNAME as $USER |
scp -p -r $USER@$HOST: file dir/ | Copy with permissions to $USER's home directory on $HOST | |
scp -c arcfour $USER@$LANHOST: bigfile | Use faster crypto for local LAN. This might saturate GigE | |
ssh -g -L 8080:localhost:80 root@$HOST | Forward connections to $HOSTNAME:8080 out to $HOST:80 | |
ssh -R 1434:imap:143 root@$HOST | Forward connections from $HOST:1434 in to imap:143 | |
ssh-copy-id $USER@$HOST | Install public key for $USER@$HOST for password-less log in | |
wget (multi purpose download tool) | ||
• | (cd dir/ && wget -nd -pHEKk http://www.pixelbeat.org/cmdline.html) | Store local browsable version of a page to the current dir |
wget -c http://www.example.com/large.file | Continue downloading a partially downloaded file | |
wget -r -nd -np -l1 -A '*.jpg' http://www.example.com/dir/ | Download a set of files to the current directory | |
wget ftp://remote/file[1-9].iso/ | FTP supports globbing directly | |
• | wget -q -O- http://www.pixelbeat.org/timeline.html | grep 'a href' | head | Process output directly |
echo 'wget url' | at 01:00 | Download url at 1AM to current dir | |
wget --limit-rate=20k url | Do a low priority download (limit to 20KB/s in this case) | |
wget -nv --spider --force-html -i bookmarks.html | Check links in a file | |
wget --mirror http://www.example.com/ | Efficiently update a local copy of a site (handy from cron) | |
networking (Note ifconfig, route, mii-tool, nslookup commands are obsolete) | ||
ethtool eth0 | Show status of ethernet interface eth0 | |
ethtool --change eth0 autoneg off speed 100 duplex full | Manually set ethernet interface speed | |
iwconfig eth1 | Show status of wireless interface eth1 | |
iwconfig eth1 rate 1Mb/s fixed | Manually set wireless interface speed | |
• | iwlist scan | List wireless networks in range |
• | ip link show | List network interfaces |
ip link set dev eth0 name wan | Rename interface eth0 to wan | |
ip link set dev eth0 up | Bring interface eth0 up (or down) | |
• | ip addr show | List addresses for interfaces |
ip addr add 1.2.3.4/24 brd + dev eth0 | Add (or del) ip and mask (255.255.255.0) | |
• | ip route show | List routing table |
ip route add default via 1.2.3.254 | Set default gateway to 1.2.3.254 | |
• | host pixelbeat.org | Lookup DNS ip address for name or vice versa |
• | hostname -i | Lookup local ip address (equivalent to host `hostname`) |
• | whois pixelbeat.org | Lookup whois info for hostname or ip address |
• | netstat -tupl | List internet services on a system |
• | netstat -tup | List active connections to/from system |
windows networking (Note samba is the package that provides all this windows specific networking support) | ||
• | smbtree | Find windows machines. See also findsmb |
nmblookup -A 1.2.3.4 | Find the windows (netbios) name associated with ip address | |
smbclient -L windows_box | List shares on windows machine or samba server | |
mount -t smbfs -o fmask=666,guest //windows_box/share /mnt/share | Mount a windows share | |
echo 'message' | smbclient -M windows_box | Send popup to windows machine (off by default in XP sp2) | |
text manipulation (Note sed uses stdin and stdout. Newer versions support inplace editing with the -i option) | ||
sed 's/string1/string2/g' | Replace string1 with string2 | |
sed 's/(.*)1/12/g' | Modify anystring1 to anystring2 | |
sed '/ *#/d; /^ *$/d' | Remove comments and blank lines | |
sed ':a; /$/N; s/n//; ta' | Concatenate lines with trailing | |
sed 's/[ t]*$//' | Remove trailing spaces from lines | |
sed 's/([`"$])/1/g' | Escape shell metacharacters active within double quotes | |
• | seq 10 | sed "s/^/ /; s/ *(.{7,})/1/" | Right align numbers |
sed -n '1000{p;q}' | Print 1000th line | |
sed -n '10,20p;20q' | Print lines 10 to 20 | |
sed -n 's/.*<title>(.*)</title>.*/1/ip;T;q' | Extract title from HTML web page | |
sed -i 42d ~/.ssh/known_hosts | Delete a particular line | |
sort -t. -k1,1n -k2,2n -k3,3n -k4,4n | Sort IPV4 ip addresses | |
• | echo 'Test' | tr '[:lower:]' '[:upper:]' | Case conversion |
• | tr -dc '[:print:]' < /dev/urandom | Filter non printable characters |
• | tr -s '[:blank:]' 't' </proc/diskstats | cut -f4 | cut fields separated by blanks |
• | history | wc -l | Count lines |
set operations (Note you can export LANG=C for speed. Also these assume no duplicate lines within a file) | ||
sort file1 file2 | uniq | Union of unsorted files | |
sort file1 file2 | uniq -d | Intersection of unsorted files | |
sort file1 file1 file2 | uniq -u | Difference of unsorted files | |
sort file1 file2 | uniq -u | Symmetric Difference of unsorted files | |
join -t'' -a1 -a2 file1 file2 | Union of sorted files | |
join -t'' file1 file2 | Intersection of sorted files | |
join -t'' -v2 file1 file2 | Difference of sorted files | |
join -t'' -v1 -v2 file1 file2 | Symmetric Difference of sorted files | |
math | ||
• | echo '(1 + sqrt(5))/2' | bc -l | Quick math (Calculate ?). See also bc |
• | seq -f '4/%g' 1 2 99999 | paste -sd-+ | bc -l | Calculate ? the unix way |
• | echo 'pad=20; min=64; (100*10^6)/((pad+min)*8)' | bc | More complex (int) e.g. This shows max FastE packet rate |
• | echo 'pad=20; min=64; print (100E6)/((pad+min)*8)' | python | Python handles scientific notation |
• | echo 'pad=20; plot [64:1518] (100*10**6)/((pad+x)*8)' | gnuplot -persist | Plot FastE packet rate vs packet size |
• | echo 'obase=16; ibase=10; 64206' | bc | Base conversion (decimal to hexadecimal) |
• | echo $((0x2dec)) | Base conversion (hex to dec) ((shell arithmetic expansion)) |
• | units -t '100m/9.58s' 'miles/hour' | Unit conversion (metric to imperial) |
• | units -t '500GB' 'GiB' | Unit conversion (SI to IEC prefixes) |
• | units -t '1 googol' | Definition lookup |
• | seq 100 | (tr 'n' +; echo 0) | bc | Add a column of numbers. See also add and funcpy |
calendar | ||
• | cal -3 | Display a calendar |
• | cal 9 1752 | Display a calendar for a particular month year |
• | date -d fri | What date is it this friday. See also day |
• | [ $(date -d '12:00 +1 day' +%d) = '01' ] || exit | exit a script unless it's the last day of the month |
• | date --date='25 Dec' +%A | What day does xmas fall on, this year |
• | date --date='@2147483647' | Convert seconds since the epoch (1970-01-01 UTC) to date |
• | TZ='America/Los_Angeles' date | What time is it on west coast of US (use tzselect to find TZ) |
• | date --date='TZ="America/Los_Angeles" 09:00 next Fri' | What's the local time for 9AM next Friday on west coast US |
locales | ||
• | printf "%'dn" 1234 | Print number with thousands grouping appropriate to locale |
• | BLOCK_SIZE='1 ls -l | Use locale thousands grouping in ls. See also l |
• | echo "I live in `locale territory`" | Extract info from locale database |
• | LANG=en_IE.utf8 locale int_prefix | Lookup locale info for specific country. See also ccodes |
• | locale -kc $(locale | sed -n 's/(LC_.{4,})=.*/1/p') | less | List fields available in locale database |
recode (Obsoletes iconv, dos2unix, unix2dos) | ||
• | recode -l | less | Show available conversions (aliases on each line) |
recode windows-1252.. file_to_change.txt | Windows "ansi" to local charset (auto does CRLF conversion) | |
recode utf-8/CRLF.. file_to_change.txt | Windows utf8 to local charset | |
recode iso-8859-15..utf8 file_to_change.txt | Latin9 (western europe) to utf8 | |
recode ../b64 < file.txt > file.b64 | Base64 encode | |
recode /qp.. < file.qp > file.txt | Quoted printable decode | |
recode ..HTML < file.txt > file.html | Text to HTML | |
• | recode -lf windows-1252 | grep euro | Lookup table of characters |
• | echo -n 0x80 | recode latin-9/x1..dump | Show what a code represents in latin-9 charmap |
• | echo -n 0x20AC | recode ucs-2/x2..latin-9/x | Show latin-9 encoding |
• | echo -n 0x20AC | recode ucs-2/x2..utf-8/x | Show utf-8 encoding |
CDs | ||
gzip < /dev/cdrom > cdrom.iso.gz | Save copy of data cdrom | |
mkisofs -V LABEL -r dir | gzip > cdrom.iso.gz | Create cdrom image from contents of dir | |
mount -o loop cdrom.iso /mnt/dir | Mount the cdrom image at /mnt/dir (read only) | |
cdrecord -v dev=/dev/cdrom blank=fast | Clear a CDRW | |
gzip -dc cdrom.iso.gz | cdrecord -v dev=/dev/cdrom - | Burn cdrom image (use dev=ATAPI -scanbus to confirm dev) | |
cdparanoia -B | Rip audio tracks from CD to wav files in current dir | |
cdrecord -v dev=/dev/cdrom -audio -pad *.wav | Make audio CD from all wavs in current dir (see also cdrdao) | |
oggenc --tracknum='track' track.cdda.wav -o 'track.ogg' | Make ogg file from wav file | |
disk space (See also FSlint) | ||
• | ls -lSr | Show files by size, biggest last |
• | du -s * | sort -k1,1rn | head | Show top disk users in current dir. See also dutop |
• | du -hs /home/* | sort -k1,1h | Sort paths by easy to interpret disk usage |
• | df -h | Show free space on mounted filesystems |
• | df -i | Show free inodes on mounted filesystems |
• | fdisk -l | Show disks partitions sizes and types (run as root) |
• | rpm -q -a --qf '%10{SIZE}t%{NAME}n' | sort -k1,1n | List all packages by installed size (Bytes) on rpm distros |
• | dpkg-query -W -f='${Installed-Size;10}t${Package}n' | sort -k1,1n | List all packages by installed size (KBytes) on deb distros |
• | dd bs=1 seek=2TB if=/dev/null of=ext3.test | Create a large test file (taking no space). See also truncate |
• | > file | truncate data of file or create an empty file |
monitoring/debugging | ||
• | tail -f /var/log/messages | Monitor messages in a log file |
• | strace -c ls >/dev/null | Summarise/profile system calls made by command |
• | strace -f -e open ls >/dev/null | List system calls made by command |
• | strace -f -e trace=write -e write=1,2 ls >/dev/null | Monitor what's written to stdout and stderr |
• | ltrace -f -e getenv ls >/dev/null | List library calls made by command |
• | lsof -p $$ | List paths that process id has open |
• | lsof ~ | List processes that have specified path open |
• | tcpdump not port 22 | Show network traffic except ssh. See also tcpdump_not_me |
• | ps -e -o pid,args --forest | List processes in a hierarchy |
• | ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu | sed '/^ 0.0 /d' | List processes by % cpu usage |
• | ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS | List processes by mem (KB) usage. See also ps_mem.py |
• | ps -C firefox-bin -L -o pid,tid,pcpu,state | List all threads for a particular process |
• | ps -p 1,$$ -o etime= | List elapsed wall time for particular process IDs |
• | last reboot | Show system reboot history |
• | free -m | Show amount of (remaining) RAM (-m displays in MB) |
• | watch -n.1 'cat /proc/interrupts' | Watch changeable data continuously |
• | udevadm monitor | Monitor udev events to help configure rules |
system information (see also sysinfo) ('#' means root access is required) | ||
• | uname -a | Show kernel version and system architecture |
• | head -n1 /etc/issue | Show name and version of distribution |
• | cat /proc/partitions | Show all partitions registered on the system |
• | grep MemTotal /proc/meminfo | Show RAM total seen by the system |
• | grep "model name" /proc/cpuinfo | Show CPU(s) info |
• | lspci -tv | Show PCI info |
• | lsusb -tv | Show USB info |
• | mount | column -t | List mounted filesystems on the system (and align output) |
• | grep -F capacity: /proc/acpi/battery/BAT0/info | Show state of cells in laptop battery |
# | dmidecode -q | less | Display SMBIOS/DMI information |
# | smartctl -A /dev/sda | grep Power_On_Hours | How long has this disk (system) been powered on in total |
# | hdparm -i /dev/sda | Show info about disk sda |
# | hdparm -tT /dev/sda | Do a read speed test on disk sda |
# | badblocks -s /dev/sda | Test for unreadable blocks on disk sda |
interactive (see also linux keyboard shortcuts) | ||
• | readline | Line editor used by bash, python, bc, gnuplot, ... |
• | screen | Virtual terminals with detach capability, ... |
• | mc | Powerful file manager that can browse rpm, tar, ftp, ssh, ... |
• | gnuplot | Interactive/scriptable graphing |
• | links | Web browser |
• | xdg-open . | open a file or url with the registered desktop application |
More Linux Commands
Examples marked with • are valid/safe to paste without modification into a terminal, so
you may want to keep a terminal window open while reading this so you can cut & paste.
Command | Description | |
• | grep . /proc/sys/net/ipv4/* | List the contents of flag files |
• | set | grep $USER | Search current environment |
• | tr '' 'n' < /proc/$$/environ | Display the startup environment for any process |
• | echo $PATH | tr : 'n' | Display the $PATH one per line |
• | kill -0 $$ && echo process exists and can accept signals | Check for the existence of a process (pid) |
• | find /etc -readable | xargs less -K -p'*ntp' -j $((${LINES:-25}/2)) | Search paths and data with full context. Use n to iterate |
Low impact admin | ||
# | apt-get install "package" -o Acquire::http::Dl-Limit=42 -o Acquire::Queue-mode=access | Rate limit apt-get to 42KB/s |
echo 'wget url' | at 01:00 | Download url at 1AM to current dir | |
# | apache2ctl configtest && apache2ctl graceful | Restart apache if config is OK |
• | nice openssl speed sha1 | Run a low priority command (openssl benchmark) |
• | chrt -i 0 openssl speed sha1 | Run a low priority command (more effective than nice) |
• | renice 19 -p $$; ionice -c3 -p $$ | Make shell (script) low priority. Use for non interactive tasks |
Interactive monitoring | ||
• | watch -t -n1 uptime | Clock with system load |
• | htop -d 5 | Better top (scrollable, tree view, lsof/strace integration, ...) |
• | iotop | What's doing I/O |
# | watch -d -n30 "nice ps_mem.py | tail -n $((${LINES:-12}-2))" | What's using RAM |
# | iftop | What's using the network. See also iptraf |
# | mtr www.pixelbeat.org | ping and traceroute combined |
Useful utilities | ||
• | pv < /dev/zero > /dev/null | Progress Viewer for data copying from files and pipes |
• | wkhtml2pdf http://.../linux_commands.html linux_commands.pdf | Make a pdf of a web page |
• | timeout 1 sleep 3 | run a command with bounded time. See also timeout |
Networking | ||
• | python -m SimpleHTTPServer | Serve current directory tree at http://$HOSTNAME:8000/ |
• | openssl s_client -connect www.google.com:443 </dev/null 2>&0 | openssl x509 -dates -noout | Display the date range for a site's certs |
• | curl -I www.pixelbeat.org | Display the server headers for a web site |
# | lsof -i tcp:80 | What's using port 80 |
# | httpd -S | Display a list of apache virtual hosts |
• | vim scp://user@remote//path/to/file | Edit remote file using local vim. Good for high latency links |
• | curl -s http://www.pixelbeat.org/pixelbeat.asc | gpg --import | Import a gpg key from the web |
• | tc qdisc add dev lo root handle 1:0 netem delay 20msec | Add 20ms latency to loopback device (for testing) |
• | tc qdisc del dev lo root | Remove latency added above |
Notification | ||
• | echo "DISPLAY=$DISPLAY xmessage cooker" | at "NOW +30min" | Popup reminder |
• | notify-send "subject" "message" | Display a gnome popup notification |
echo "mail -s 'go home' P@draigBrady.com < /dev/null" | at 17:30 | Email reminder | |
uuencode file name | mail -s subject P@draigBrady.com | Send a file via email | |
ansi2html.sh | mail -a "Content-Type: text/html" P@draigBrady.com | Send/Generate HTML email | |
Better default settings (useful in your .bashrc) | ||
# | tail -s.1 -f /var/log/messages | Display file additions more responsively |
• | seq 100 | tail -n $((${LINES:-12}-2)) | Display as many lines as possible without scrolling |
# | tcpdump -s0 | Capture full network packets |
Useful functions/aliases (useful in your .bashrc) | ||
• | md () { mkdir -p "$1" && cd "$1"; } | Change to a new directory |
• | strerror() { python -c "import os; print os.strerror($1)"; } | Display the meaning of an errno |
• | plot() { { echo 'plot "-"' "$@"; cat; } | gnuplot -persist; } | Plot stdin. (e.g: • seq 1000 | sed 's/.*/s(&)/' | bc -l | plot) |
• | hili() { e="$1"; shift; grep --col=always -Eih "$e|$" "$@"; } | highlight occurences of expr. (e.g: • env | hili $USER) |
• | alias hd='od -Ax -tx1z -v' | Hexdump. (usage e.g.: • hd /proc/self/cmdline | less) |
• | alias realpath='readlink -f' | Canonicalize path. (usage e.g.: • realpath ~/../$USER) |
• | ord() { printf "0x%xn" "'$1"; } | shell version of the ord() function |
• | chr() { printf $(printf '%03on' "$1"); } | shell version of the chr() function |
Multimedia | ||
• | DISPLAY=:0.0 import -window root orig.png | Take a (remote) screenshot |
• | convert -filter catrom -resize '600x>' orig.png 600px_wide.png | Shrink to width, computer gen images or screenshots |
mplayer -ao pcm -vo null -vc dummy /tmp/Flash* | Extract audio from flash video to audiodump.wav | |
ffmpeg -i filename.avi | Display info about multimedia file | |
• | ffmpeg -f x11grab -s xga -r 25 -i :0 -sameq demo.mpg | Capture video of an X display |
DVD | ||
for i in $(seq 9); do ffmpeg -i $i.avi -target pal-dvd $i.mpg; done | Convert video to the correct encoding and aspect for DVD | |
dvdauthor -odvd -t -v "pal,4:3,720xfull" *.mpg;dvdauthor -odvd -T | Build DVD file system. Use 16:9 for widescreen input | |
growisofs -dvd-compat -Z /dev/dvd -dvd-video dvd | Burn DVD file system to disc | |
Unicode | ||
• | python -c "import unicodedata as u; print u.name(unichr(0x2028))" | Lookup a unicode character |
• | uconv -f utf8 -t utf8 -x nfc | Normalize combining characters |
• | printf '300200' | iconv -futf8 -tutf8 >/dev/null | Validate UTF-8 |
• | printf '?TF8n' | LANG=C grep --color=always '[^ -~]+' | Highlight non printable ASCII chars in UTF-8 |
• | fc-match -s "sans:lang=zh" | List font match order for language and style |
Development | ||
• | gcc -march=native -E -v -</dev/null 2>&1|sed -n 's/.*-mar/-mar/p' | Show autodetected gcc tuning params. See also gcccpuopt |
• | for i in $(seq 4); do { [ $i = 1 ] && wget http://url.ie/6lko -qO-|| ./a.out; } | tee /dev/tty | gcc -xc - 2>/dev/null; done | Compile and execute C code from stdin |
• | cpp -dM /dev/null | Show all predefined macros |
• | echo "#include <features.h>" | cpp -dN | grep "#define __USE_" | Show all glibc feature macros |
gdb -tui | Debug showing source code context in separate windows | |
udev | ||
• | udevadm info -a -p $(udevadm info -q path -n /dev/input/mouse0) | List udev attributes of a device, for matching rules etc. |
• | udevadm test /sys/class/input/mouse0 | See how udev rules are applied for a device |
# | udevadm control --reload-rules | Reload udev rules after modification |
Extended Attributes (Note you may need to (re)mount with "acl" or "user_xattr" options) | ||
• | getfacl . | Show ACLs for file |
• | setfacl -m u:nobody:r . | Allow a specific user to read file |
• | setfacl -x u:nobody . | Delete a specific user's rights to file |
setfacl --default -m group:users:rw- dir/ | Set umask for a for a specific dir | |
getcap file | Show capabilities for a program | |
setcap cap_net_raw+ep your_gtk_prog | Allow gtk program raw access to network | |
• | stat -c%C . | Show SELinux context for file |
chcon ... file | Set SELinux context for file (see also restorecon) | |
• | getfattr -m- -d . | Show all extended attributes (includes selinux,acls,...) |
• | setfattr -n "user.foo" -v "bar" . | Set arbitrary user attributes |
BASH specific | ||
• | echo 123 | tee >(tr 1 a) | tr 1 b | Split data to 2 commands (using process substitution) |
meld local_file <(ssh host cat remote_file) | Compare a local and remote file (using process substitution) | |
Multicore | ||
• | taskset -c 0 nproc | Restrict a command to certain processors |
• | find -type f -print0 | xargs -r0 -P$(nproc) -n10 md5sum | Process files in parallel over available processors |
sort -m <(sort data1) <(sort data2) >data.sorted | Sort separate data files over 2 processors |
Thursday, 10 May 2012
How to restore Admin access for failed TLS/SSL access with phpMyAdmin
1. Open your core_config_data table in phpMyAdmin.
2. Find the row with the path web/secure/use_in_adminhtml and change its value field from 1 to 0 to enable accessing admin panel from unsecure http://www.yourwebsite.com/admin url
Changing web/secure/use_in_frontend toggles customer shopping cart security, 1=on and 0=off which probably isn't of importance as you're trying to regain administrative access
3. Clear /var/cache, /var/session and after you've done the above and regained access your system, reindex yourURL_rewrite index after changing settings. This is necessary because your config is cached and clearing it forces a reread of the configuration data from the core_config_data table.
You should now be able to access your Magento Admin panel by standard unsecured web access (port 80, http).
NOTE: Be aware that accidentally entering your leading https:// instead of http:// on your unsecure_base_url before you have enabled TLS/SSL on your webserver will lock you out, so if the above doesn't get you in, look for rows withweb/unsecure/base_url and check for the aforementioned misconfiguration
How to fix the 10 most common Magento problems
- How to configure Magento to work with a new domain;
- How to reset the Magento admin password;
- How to enable SEF URLs in Magento;
- How to speed up Magento;
- How to redirect Magento to open through www;
- How to disable the Compare products functionality;
- How to set up a blog in Magento;
- How to add a Contact Us form in Magento;
- How to fix the ''Access Denied'' message in the Magento admin area;
- How to set a custom group of users with a discount in a Magento store;
How to configure Magento to work with a new domain
There are two things you should do in order to configure Magento to work with a new domain:
- Edit the Magento database
Go to your cPanel > phpMyAdmin. Select your Magento database from the left menu, find the table called core_config_data and click on it.
Click the Browse tab and edit the first two fields:
web/unsecure/base_url
web/secure/base_url
by clicking the pen icon in front of each of them. Replace your old domain name with your new one and click the Go button to save the change.
- Clear the Magento cache.
The Magento cache folder is located in your Magento installation directory > /var/cache. To clear the cache, simply delete the folder.
Many Magento issues can be fixed just by deleting the cache.
How to reset Magento Admin Password
To change your Magento admin password, go to your cPanel > phpMyAdmin, select your Magento database, click the SQL tab and paste this query:
UPDATE admin_user SET password=CONCAT(MD5('sGnewpass'), ':sG') WHERE username='AdminUsername';
Note: You have to change newpass in the MD5('sGnewpass') with your new password, and change *AdminUsername* to your Magento admin username.
Execute the query by clicking the Go button and your password will be changed.
How to enable Search Engine Friendly URLs in Magento
To enable Search Engine Friendly URLs in Magento, you have to log in to the Magento administration area and click on theConfiguration button. Under the System navigation menu, switch to Web page from the sub-navigation panel on the left.
When the page loads, you will see blue lines which represent closed options tablets. Click on the Search Engines Optimization tab and turn on the Use Web Server Rewrites (mark as Yes). Click on the Save Config button and your Magento SEF URLs will be enabled.
How to speed up Magento
Many Magento issues are caused by slow performance. The recommended way to speed up Magento's performance is to enable itsCompilation function. The performance increase is between 25%-50% on page loads.
You can enable Magento Compilation from your Magento admin panel > System > Tools > Compilation.
How to redirect Magento to open through www
For SEO and usability purposes you may want to redirect your visitors to open your site only through www (http://www.yourdomain.com).
To do this in Magento, you should open the .htaccess file in the folder where your Magento is installed. In it locate theRewriteEngine on line and right after it add the following lines:
RewriteCond %{HTTP_HOST} ^yourdomain.com$ [NC]
RewriteRule ^(.*)$ http://www.yourdomain.com/$1 [R=301,L]
Once you do this, save the .htaccess file and log in to the Magento admin area > System > Configuration menu and from the left panel click the Web button.
Unfold the Unsecured set of options and change the Base URL option from http://yourdomain.com to http://www.yourdomain.com.
Save the changes and your Magento will start working through www.yourdomain.com only!
How to disable the Compare products functionality
You can disable the Compare products functionality in Magento by following these steps:
- Edit app/code/core/Mage/Catalog/Helper/Product/Compare.php and change the following code:
public function getAddUrl($product)
{
return $this->_getUrl(’catalog/product_compare/add’, $this->_getUrlParams($product));
}
to
public function getAddUrl($product)
{
//return $this->_getUrl(’catalog/product_compare/add’, $this->_getUrlParams($product)); return false;
}
- Edit ./app/design/frontend/base/default/layout/catalog.xml (if you are using a different Magento theme, enter its name instead of default) and change the following code:
<block type="catalog/product_compare_sidebar" before="cart_sidebar" name="catalog.compare.sidebar" template="catalog/product/compare/sidebar.phtml"/>
to
<!-- <block type="catalog/product_compare_sidebar" before="cart_sidebar" name="catalog.compare.sidebar" template="catalog/product/compare/sidebar.phtml"/> -->
- Flush the Magento cache from your Magento admin area > System > Cache Management.
How to set up a blog in Magento
It is not difficult to set up a blog in Magento. However, note that this functionality is not included by default and you will have to use a custom extension to add it.
You can search Magento Connect for an extension that will fully suit your needs. One of the popular free extensions that you can use is the Magento Blog – Community Edition.
All Magento extensions are installed in a similar way that is thoroughly explained in our Magento Connect Tutorial.
Once the extension is installed, you will have one additional section in the top menu of your Magento admin area called Blog. From there you can adjust the newly-installed Blog settings, add posts etc.
How to add a Contact Us form in Magento
Magento includes contact form functionality by default. A link to a contact form can usually be found in the footer of your Magento installation.
Of course, you can add a contact form on any page. All you need to do is:
- Log in to the administrator area.
- Go to CMS > Pages.
- Select the page you want to edit or create a new page.
Paste the following code using the HTML option of the WYSIWYG editor:
<!– CONTACT FORM CODE BEGIN–>
{{block type='core/template' name='contactForm' template='contacts/form.phtml'}}
<!– CONTACT FORM CODE END–>
Save the changes and the contact form will appear on the desired page.
"Access denied" issue
As a solution to the "Access denied" issue, you should log out from the Magento admin area and then log in again.
If the above does not help, you should reset the admin privileges. This can be done through the Magento admin area > System > Permissions > Roles > Administrators.
Click on the Role Resources option from the left menu and make sure that Resource Access is set to All.
Click on the Save Role button and the permissions will be reset.
How to set a custom group of users
You can add a new group from the Magento admin area > Customers > Customer Groups > Add New Customer Group.
Once a customer registers, you can change the group he/she belongs to from the Magento admin area > Customers > Manage Customers. Click on the Edit link next to the customer and change the group from the Account Information > Customer Group. Click Save Customer.
Set the discount from Promotions > Catalog Price Rules > Add New Rule.
In the Customer Groups select the customers' groups for which the promotion is valid. Enter the other details, set the rule actions and conditions. Finally, click Save Rule.
The above ten tips will hopefully help you resolve at least some of the Magento issues you have faced or are about to face.