In preparation for this, you will need to install yaourt and enable it with sudo access. Either that or replace all the references to yaourt below with pacman.

Drop this into your ~/.bashrc (or zshrc):

function arch() {
        if [ -z "$1" ]; then
                echo "arch  " >&2
                return -1
        fi

        case $1 in
                upgrade)
                        # Synchronize, upgrade
                        shift
                        sudo yaourt -Syu $@
                        ;;
                install)
                        # Install stuff
                        shift
                        sudo yaourt -S $@
                        ;;
                localinstall)
                        # Install file
                        shift
                        sudo yaourt -U $@
                        ;;
                remove)
                        # Remove
                        shift
                        sudo yaourt -R $@
                        ;;
                fullremove)
                        # Remove file
                        shift
                        sudo yaourt -Rns $@
                        ;;
                info)
                        # Display information
                        shift
                        yaourt -Si $@
                        ;;
                search)
                        # Search
                        shift
                        yaourt -Ss $@
                        ;;
        esac
}

Now you can do things like:

$ arch install git
$ arch remove git
$ arch info git
$ arch search git

Hope its handy!

Lets see how SSH works without any setup:

$ ssh znx@work.host.com
 znx@work.host.com's password: (password)

Type in your password and you get a shell. Now lets generate some keys and login with those.

$ cd ~/.ssh
$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/mark/.ssh/id_rsa): id_workhost
Enter passphrase (empty for no passphrase): (passphrase)
Enter same passphrase again: (passphrase)
Your identification has been saved in id_workhost.
Your public key has been saved in id_workhost.pub.
The key fingerprint is:
75:9c:c0:a3:52:7e:92:3c:f7:9d:c0:ff:0b:18:8c:48 znx@home.local
The key's randomart image is:
+--[ RSA 2048]----+
|         ..      |
|        . oo .   |
|       E o.o+    |
|      o O.=.o    |
|       oS* + + . |
|            + +  |
|           . . . |
|              . .|
|               ..|
+-----------------+

So the private key is called id_workhost and the public key is called id_workhost.pub. The private key should be kept safe at all times; the public key however is what we need to put out onto the other machine. If you have ssh-copy-id you can use that instead of the follow. So lets do that now:

$ scp ~/.ssh/id_workhost.pub znx@work.host.com:
znx@work.host.com's password:  (password)
id_workhost.pub                                100% 1766     1.7KB/s   00:00
$ ssh znx@work.host.com
znx@work.host.com's password:  (password)
work$ cat id_workhost.pub >> ~/.ssh/authorized_keys
work$ rm id_workhost.pub
work$ logout
$ ssh -i ~/.ssh/id_workhost znx@work.host.com
Enter passphrase for key '/home/znx/.ssh/id_workhost': (passphrase)
Last login: Wed Apr  6 08:25:16 2011 from home.local
work$

So we are now using keys to login rather than passwords but it still requires a passphrase to be type in. The next step is to look at ssh-agent. This tool will hold keys in memory and allow thus allowing you to login without needing to unlock the key. So lets run it and see what it does:

$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-SdjNGn3604/agent.3604; export SSH_AUTH_SOCK;
SSH_AGENT_PID=3605; export SSH_AGENT_PID;
echo Agent pid 3605;

Not exactly useful until you spot that it is actually setting up bash variables. So the way we want to run ssh-agent is as follows:

$ eval `ssh-agent`
Agent pid 3609
$ echo $SSH_AGENT_PID
3609

That’s more like it, so once you have a running agent you can now start to add keys into its memory.

$ ssh-add ~/.ssh/id_workhost
Enter passphrase for /home/mark/.ssh/id_workhost: (passphrase)
Identity added: /home/mark/.ssh/id_workhost (/home/mark/.ssh/id_workhost)

With the key added we can now login to the host with the unlocked key:

$ ssh znx@work.host.com
Last login: Wed Apr  6 22:14:49 2011 from home.local
work$

Now that is a login without a password or a passphrase because the key is unlocked and held by ssh-agent. Whilst ssh-agent is useful if you retain one shell open all the time, the moment you close that shell you lose the variables of the running ssh-agent which means that you will not be able to use it. This also means that if you are running scripts (say via cron) that you can no longer automate the logins. So back to keys without passphrase? NO!

There is a very nice tool that can help us out and it is called keychain. Simply add this to your ~/.bash_profile:

keychain --quiet ~/.ssh/id_workhost
source ~/.keychain/*-sh

You should also add this to your ~/.bashrc:

source ~/.keychain/*-sh

Now login to your shell again and you should see that you are prompted to put in passwords. This is keychain loading your keys in. It looks something like this:

Identity added: /home/mark/.ssh/id_mine (/home/mark/.ssh/id_mine)

You will see that you can now login without passphrase to your host again. Indeed all the shells you open will now grab the keychain information and login immediately.

This means that you will only get prompted for passphrases once when you login and then you will have the ability to login to your shells without passphrases. Obviously you should add keys for your regular hosts. You can add keys on the fly with keychain at the command line:

$ keychain ~/.ssh/id_someotherkey

And it will be stored until your ssh-agent is closed. If you wish to clear the keys you have added then simply do:

$ keychain --clear

What all this means is that you can now run scripts with passphrase protected keys because all you need to do is source the running ssh-agent within your script:

#!/bin/sh
# Some remote copy script
source ~/.keychain/*-sh

scp znx@work.host.com:nightly-copy.file local/nightly-copy.file

As long as the key is load the script will work.

Doing this ensures that even if the private key falls into the wrong hands that it is still secured with a passphrase and you haven’t just opened up your remote hosts to attack.

There is solutions for this under Windows as well; the PuTTY installer will provide you with a tool called Pageant. This works in an identical way to ssh-agent; loading keys into its memory to allow PuTTY to login with unlocked keys.

Check out some of the other SSH tips I’ve published which will ease the logins even further!

[user@host]$ ssh test -l user
user@test's password:
[user@test]$ ulimit -n
1024
[user@test]$ cat /etc/security/limits.conf
* soft nofile 4096
* hard nofile 4096

However switching to the same user you will see:

[user@test]$ su - user
Password:
[user@test]$ ulimit -n
4096

That the limit isn’t respect when logging in via SSH but when we switch user it is applied.

The reason for this is rather simple; the SSH is opening a shell that isn’t a login shell and therefore limits is not being applied. To correct this simply edit your sshd_config file and set it to use login shells.

[root@test]$ grep -i uselogin /etc/ssh/sshd_config
#UseLogin no

You can see that the entry is set by default to no; so simply edit the line and alter it to yes:

[root@test]$ sed -i.bak 's/#UseLogin no/UseLogin yes/' /etc/ssh/sshd_config
[root@test]$ grep -i uselogin /etc/ssh/sshd_config
UseLogin yes

Now reload your sshd.

[root@test]$ /etc/init.d/sshd reload
 /etc/init.d/sshd reload
Reloading sshd:                                            [  OK  ]

And test again:

[user@host]$ ssh test -l user
user@test's password:
[user@test]$ ulimit -n
4096

That’s it! Sometimes odd behaviour like this can be difficult to track down without having a good read of configuration files.

unrar t file.rar
if [ $? -eq 0 ]; then
  echo "Rar file is good?"
fi

However it was returning zero all the time, even when the file wasn’t a rar:

# unrar t file.rar
 
UNRAR 3.80 freeware      Copyright (c) 1993-2008 Alexander Roshal
 
file.rar is not RAR archive
# echo $?
0

So it fails the test but returns zero regardless. This makes it very unhelpful for using in scripting. Fortunately enough a mate on IRC discovered that his version did.

So I first download the existing SRPM and installed it:

# yumdownloader --source unrar
# rpm -i unrar*.srpm

Then I installed that and simply modified so I downloaded the latest, created a RPM and installed.

I have submitted the updated spec file to RPMfusion.

[root@marine]# ls
adobe-release-i386-1.0-1.noarch.rpm
[root@marine]# yum localinstall adobe-release-i386-1.0-1.noarch.rpm
Loaded plugins: fastestmirror, priorities
Setting up Local Package Process
Examining adobe-release-i386-1.0-1.noarch.rpm: adobe-release-i386-1.0-1.noarch
Marking adobe-release-i386-1.0-1.noarch.rpm to be installed
Loading mirror speeds from cached hostfile
* addons: anorien.csc.warwick.ac.uk
* base: anorien.csc.warwick.ac.uk
* centosplus: anorien.csc.warwick.ac.uk
* contrib: anorien.csc.warwick.ac.uk
* extras: anorien.csc.warwick.ac.uk
* rpmforge: fr2.rpmfind.net
* updates: anorien.csc.warwick.ac.uk
588 packages excluded due to repository priority protections
Resolving Dependencies
--> Running transaction check
---> Package adobe-release-i386.noarch 0:1.0-1 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
Package              Arch     Version Repository                          Size
================================================================================
Installing:
adobe-release-i386   noarch   1.0-1   /adobe-release-i386-1.0-1.noarch   1.9 k

Transaction Summary
================================================================================
Install      1 Package(s)
Update       0 Package(s)
Remove       0 Package(s)

Total size: 1.9 k
Is this ok [y/N]: y
Downloading Packages:
warning: rpmts_HdrFromFdno: Header V3 DSA signature: NOKEY, key ID f6777c67

Public key for adobe-release-i386-1.0-1.noarch.rpm is not installed

As you can see it attempted to install but because the key was missing it refused to install. So what do we do to find that key? First we must download the key from somewhere, there is various key servers around the world, some of the most popular are (note hkp is the HTTP keyserver protocol):

• hkp://subkeys.pgp.net
• hkp://pgp.mit.edu
• hkp://keys.gnupg.net
• hkp://wwwkeys.uk.pgp.net (where UK can be replaced by any country code)

So lets search for the missing key:

[root@marine]# gpg --keyserver hkp://wwwkeys.uk.pgp.net --recv-keys f6777c67
gpg: requesting key F6777C67 from hkp server wwwkeys.uk.pgp.net
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: key F6777C67: public key "Adobe Systems Incorporated (Linux RPM
  Signing Key) " imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg:               imported: 1

So now we have the key and see that it is indeed Adobe’s key. Now all we need to do is add into RPM:

[root@marine]# gpg --armor --export f6777c67 >tmp-gpg.key
[root@marine]# rpm --import tmp-gpg.key

Simple as that, the install will work now!

This post is a refreshed look at a previous post on the same matter.

(Orlando) sed regex kicks my ass.  I like to remove the second : in the
line. So that in 123:456:6789 it will only returns 123:456.  I can find the
first :, but I have not been able to use s/:{2}// to find the second
one, and remove the rest.

I enjoy regex (I know I’m weird, leave me alone), so I was able to provide an answer to this problem:

(@znx) Orlando: the trick is to use [^:] and \1 ..
(@znx) like:   s/\(:[^:]*\):.*/\1/

Now obviously at first glance this regex could be a bit intimating to someone who is still picking up the skills but as with most things if we break it down it becomes easier.

Working out to in, \( \):.*, that says match something with : at the end and all the character after it. The . is a special meaning “any character” and * to match multiple characters. The first match will be stored by sed and assigned into the \1 for the replacement (that is what the brackets do). Inside the brackets we have :[^:]*. The sequence [^ ] is a negated list, that means that we are asking it to match everything that is NOT inside the list, in this case :.

Putting it altogether we are saying: Match a leading : and a trailing : with any characters after it. Placing the contents between the two : in memory. Then finally we replace the contents.

% echo 123:456:6789 | sed 's/\(:[^:]*\):.*/\1/'
123:456

Sucess, however as with most things, there is more than one way to skin a cat and regex is rarely the prettiest method. So what other ways can we solve this problem?

With AWK:

% echo 123:456:6789 | awk -F: '{print $1":"$2}'
123:456

With cut:

% echo 123:456:6789 | cut -d: -f1,2
123:456

As always, experimentation with the mass of GNU tools you can find on your system will bring a greater deal of power to your tool chest. Mind you, then I wouldn’t get complements for helping would I?

(Orlando) Wow, When I grow up, I like to remember this thing like you do.  

Haha, till next time!

<html><head><title>Status</title></head>
<table>
<tr><td>Failed</td><td>Backup Group</td></tr>
<tr><td>Success</td><td>Another Backup Group</td></tr>
</table>
</body>
</html>

Nothing odd there, the file is normal but when I tried this command:

$ grep -i failed status.html
$

Huh? No output, suggesting that there is no lines with the words failed on them. The same occurs with awk and sed, indeed I could not find any tool to be able to grep out the status. So the next step was to check what was odd about the file:

$ file status.html
status.html: HTML document text

Still, nothing unusual. So now I am in full head scratching mode, I open up the file with vim to see if I can discover anything strange about the file but nothing. At this point I happened to switch to more rather than cat and the result was the start of how I solved it.

$ more status.html
��<

$

Ah, so now I can see why grep and the others cannot view anything in the file. So this time I switch to vim again and check what file encoding we have:

:set fileencoding
fileencoding=ucs-2le

For those that are unaware, this is UCS-2 (little endian), also know as UTF-16. So the issue was simply that we had UTF-16 characters, now for the trick to get around it:

$ iconv -f UTF-16 -t UTF-8 status.html | grep -i failed
<tr><td>Failed</td><td>Backup Group</td></tr>
$

Tada. Once more a solution!