Linux Package management overview

I have to confess that delegating software installation to Debian and Ubuntu’s apt command is what finally converted me to Linux.  I stillhave a bias against .rpms and building from source based on disastrous experiences hunting down obscure .rpms or figuring out why make would not work.  If you’re trying out Ubuntu or another Linux distribution, you should stop and read download squad’s Package management 101

Package management refers to the way your distribution installs and configures (as well as manages and removes) software applications and libraries on your system. When Windows installs an .exe (which is the closest thing in Windows to a package) it usually places it in a single specific place within a directory. Linux installs across a few directories, leaving many new Linux users scratching their heads as to where their .rpm actually went. Most distributions install the executables in /usr/bin, and the libraries in /usr/lib. You may notice related files in /usr/share or /etc.

In short, you’ll want to let your package manager install and upgrade new software for you.  You don’t have to take my word for it, Thank You, Aptitude!

I’ve long believed that the easiest way to install software on a modern operating system is through a well-designed package manager connected to one or more carefully-maintained package repositories.

Mail Log Funnies

(02:07:13 PM) Oscar: (02:04:26 PM) Mike Shade: lol @ this:
(02:04:27 PM) Mike Shade: Sep 10 05:04:55 www sendmail[7421]: l8A94tTP007419: to=<jamseATviagra-mailDOTcom>, ctladdr=<apache@***********> (48/48), delay=00:00:0
0, xdelay=00:00:00, mailer=esmtp, pri=34248, relay=mail.viagra-mail.com. [207.44.136.131], dsn=4.0.0, stat=Deferred: 421 Too many co
ncurrent SMTP connections; please try again later.
(02:05:14 PM) Oscar: pls contact your sys-admin if viagra-mail is up for 4 or more hours at a time
(02:05:43 PM) Mike Shade: lol
(02:07:44 PM) Jo R.: hehe

Filter mail log for bounces

I needed a simple way to find hard bounces from a mail log yesterday.  Usually, you hook up a script to listen for bounced messages at an email address like "list-bounces@example.com".  The script receives an email message and parses it to figure out if and why it bounced.  I decided to take the opposite approach, and look for email addresses causing bounces in postfix’s mail log.  With Jo’s help, we figured out we could pipe two grep commands to extract the likely offenders (see below).  The key is the second grep, the -P switch looks for matches using a Perl regular expresion, and then the -v switch inverts the test. Another script can then look through the output.

grep &quot;mailer=esmtp&quot; /var/log/maillog | grep -v -P &quot;stat=(Sent|Deferred)&quot; &gt; likely-bounces.log<br />

Open source helps Nokia cut development costs

Nokia’s N770 and N800 tablet PCs were developed with a conscious decision to use as much open source software as feasible.  This decision had tremendous benefits in cost savings, code quality, flexibilty, and zero licensing costs.  You can read the white paper    HT: Stephesblog.

If we compare the code from open source to the code developed by us, our conclusion is that open source is of better quality. We have more bugs and problems in the Nokia developed code. This is only natural because the majority of the Nokia code is build from scratch and is thus very young. Open source code, on the other hand, has mostly been used by others already. They have fixed the most severe errors already before we started to use the code.

HOWTO: Convert mysql data to UTF-8 quickly

I’ve been revamping my Bolivian soccer blog, which I finally updated and made live tonight.  In doing so, I needed to convert the data from ISO-8859-1 (back when I didn’t know better about how to setup a database) to UTF-8.  I was fearing the process would be complicated and tedious, involving some script to read the data and use PHP’s mbstring or iconv extensions to do the work.  Thanks to a post on a Mysql message board (I wish I’d saved the link, I’d give you full credit), it turned out to be a drop dead simple, or is it quick-and-dirty, 3 step process at the command line using the iconv tool.

$ mysqldump --opt --quote-names my_database &gt; my_database.sql<br />$ iconv -f ISO_8859-1 -t UTF-8 my_database.sql &gt; my_database-utf-8.sql<br />$ mysql my_database &lt; my_database-utf-8.sql<br />

Why Linux?

The 3 practical reasons I use linux – the $200-400 i don’t have to give to Microsoft are used to get better hardware, more secure without blaming the user, all the Free software available for it (see the first reason again), and not HAVING to reinstall the OS quarterly.  In case that doesn’t convince you, a cavalcade of links work investigating follows.

  • Wil Whaeton- yet another linux post: points to an article to help choose a distribution, and a tutorial on using Ubuntu 6.10 to replace your MS desktop.
  • Ecogeek – Linux Prevents Obsolescence: get more CPU cycles out of that older hardware.  The lifespan, of hardware refresh, period of a Linux system is 6-8 years, compared 50 3-4 for Microsoft.  That also explains why I have the same computer at work for the last 4 years.  Plus, its good for the environment.
  • Steve Wild – Linux Truths, Halt-truths, and Myths: is Linux difficult to setup or software hard to install?  Plus an honest look at other common myths.
  • Howtoforge – The Perfect Desktop:  step-by-step guide to replacing Windows with Ubuntu, including installing common software.

Fixing Subversion Propfind 403 errors

I’ve been using Subversion heavily lately to keep my live and development sites synchronized and its been a huge productivity booster.  I had to move some code over to a lite site for work this evening and it was such a pain, because I had to go through and figure out the files I needed to copy.  With svn, or even cvs, I could use tags and then an update and let the computer do the grunt work.  Instead it was a 15 minute task and i broke the site twice due to missing dependencies.

Then, I wanted to do a quick update to soccerblogs.net, and needed to pull in code from another project within the repository using the svn:externals property.  I set the property and committed it, then when I tried to update my local development copy, the svn client would fail.  I had the following lines in my apache error log:

[Wed Feb 14 00:20:32 2007] [error] [client 65.23.154.104] client denied by server configuration: &lt;PATH TO SVN&gt;

The svn client would complaing about "PROPFIND: 403 Unauthorized".  I triple cheked my apache+svn configuration, and I could browse my repository just fine in a web browser.  It turns out that if you have mod_evasive installed, the access pattern from svn can look like a denial-of-service attack to it, so it blocks with a 403.  Disabling mod_evasive did the trick – and I didn’t really need it anyway.  None of the English language google results mentioned this, but this one in Spanish had it.  Sometimes, being bilingual pays off.

Fail2Ban + Mod_Security = Spammer Bouncer

Under debian, fail2ban’s configuration is in /etc/fail2ban/.  In the filter.d directory add the following file and name it apache-modsec.conf.

[Definition]<br /># Option:&nbsp; failregex<br /># Notes.:&nbsp; regex to match the password failure messages in the logfile. The<br />#&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; host must be matched by a group named "host". The tag "&lt;HOST&gt;" can<br />#&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; be used for standard IP/hostname matching.<br /># Values:&nbsp; TEX<br />#<br />failregex = [[]client &lt;HOST&gt;[]] mod_security: Access denied with code 500<br /># Option:&nbsp; ignoreregex<br /># Notes.:&nbsp; regex to ignore. If this regex matches, the line is ignored.<br /># Values:&nbsp; TEXT<br />#<br />ignoreregex = 

Enable this filter by  adding the followin to jail.local:

[apache-modsec]<br />enabled = true<br />port&nbsp;&nbsp;&nbsp; = http<br />filter&nbsp; = apache-modsec<br />logpath = /var/log/apache*/*error.log<br />maxretry = 4

Restart fail2ban, next time a spammer gets blocked by modsecurity 4 times, it will be recorded in /var/log/fail2ban.log

2007-02-07 11:52:45,024 fail2ban.actions: WARNING [apache-modsec] Ban 207.234.131.237