Apache: Allowing access to subpaths

Seems this is always a tricky one that I run into when setting up a new test site. If you want to lock down the entire site behind a password, but need to allow some unauthenticated requests for external integration, then have a look at the solution below. It uses SetEnvIf to open up access to selected paths. This should work too, if your route most requests through a front controller script using a rewrite rule.

In this brief tutorial, we are going to enable users to access any file or directory of a site that is password-protected via htaccess. There are many reasons for wanting to employ this technique, including:

From: Stupid htaccess Trick: Enable File or Directory Access to Your Password-Protected Site : Perishable Press

Link: Nginx Hacking Tips

This is a good overview of common configurations comparing nginx's syntax to apache's.  Once you get used to it, I think nginx syntax is easier to understand, and it reads more like an actual script or program.  I've been using nginx to host SoccerBlogs.net for 3 months now and its been very solid.  In fact, it coupled with moving mysql to its own slice, forced me to focus some attention on SQL query bottlenecks.

Nginx has a major limitation in rewrite rules in that you cannot impose multiple conditions for a rewrite rule. Apache HTTPD on the other hand provides a good solution using multiple RewriteCond directives. Nginx on the other hand allows if statement. You can have rewrite rules within if blocks. However the if block themselves are limited. You do not have and or or to add multiple conditions to a single if block. Also you cannot nest if blocks. There are no else statement either. However you can use regular expressions so the following is possible:

Nginx Hacking Tips

Using RewriteMap for many URL redirects.

Migrating platforms can lead to broken links on your website.  If you’ve cultivated links from other sites or if search engines send a lot of visitors your way, you’ll want to redirect users from the old links to the new.  Many content management systems have facilities for manually maintaining redirects, but you can also do this with Apache.  The big benefit of using Apache is that such visitors won’t hit your CMS until they get to the correct page.

One way to redirect users is through mod_rewrite and clever use of its Rewrite Rules. If you have a lot links to redirect, you don’t want to create one rule for each link, since its essentially the same pattern for each link.  This is where the rewrite modules RewriteMap directive comes in handy.

In your VirtualHost or Server sections, add something like the following, making sure to change the rewrite rule regular expression to match your own situation.


RewriteEngine On
RewriteMap redirect_map txt:/path/to/mysite/redirect_map.txt
RewriteRule ^/news/detail/(.*) ${redirect_map:$1} [R=permanent,L]

Then, in the redirect_map.txt file, you can put the old link and new link one per line and separated by one or more spaces or tabs.


1877-avoiding_frustration_with_php_sessions     /avoiding-frustration-with-php-sessions
1877-avoiding_frustration_with_php_session      /avoiding-frustration-with-php-sessions
2062-howto_use_virtualbox_to_setup_an_internet_explorer_testing_machine /howto-use-virtualbox-to-setup-an-internet-explorer-testing-machine
2067-peter_wilts_pillars_of_management          /peter-wilts-pillars-of-management

Thinking of switch to PHP w/Fast CGI?

If you’re contemplating switching to running PHP5 under FastCGI, to take advantage of apache’s threaded worker model and improve your server’s performance, think again.  If you’re used to setting and overriding php values via .htaccess files or using php_value/php_flag in your virtual host directory, this is not supported with FastCGI.  You’ll have to figure out how to set those values via some other means.  Some PHP ini directives can be set in your script directly, with ini_set, but others have to be set outside of it (register_globals for example).  There are ways to set them, but making the switch from prefork to worker is not trivial, especially if you have multiple PHP applications on a single server.

Why we like php-fastcgi and its flexibility

So if you’d like to use php-fcgi but still need some flexibility with your php configuration, (since the php-fcgi scripts are just wrappers) you can create php.ini files for whatever feature-set you need, duplicate the wrapper scripts, defining different PHPRC values for each, then use a .htaccess in each of the subfolders to define which one of the wrappers to use with the following directives.

 

Don’t abuse PHP’s header function for redirects

PHP’s Header function can come in quite handy when you’re building your next greate web application.  Its powerful, but as a result, its tempting to misuse it to do even the simplest things, like permanent redirects.  Usually, its done like this:

// redirect /publications (this page) to real page (/documents)<br />Header(&quot;Location: /documents/&quot;);<br />

One line of code, time to move on to the next task in your queue, right?   No.  Why is this wrong or wasteful?  We’re making the web server fire up the PHP processor to do something it can do more efficiently.  If we avoid using PHP, our request should be served faster and with less memory usage.  If you’re on Apache, then the Rewrite module can do this for you, assuming your hosting provider has it enabled and allows you to control it via .htaccess files.  The better way to do this is with the following:

RewriteEngine On<br />RewriteRule /publications/?&nbsp;&nbsp;  /documents/&nbsp;&nbsp;  [R=permanent, L]<br />

The "permanent" part is important, in that it’ll trigger well-behaved spiders and user-agents, like Google, to update their link database and avoid the redirect in the future altogether.  If you have a lot of redirects, this solution excels for its maintainability as well.  Keeping them all in a single .htaccess file will save you from having to hunt through multiple PHP files for that header() call.

Fixing Subversion Propfind 403 errors

I’ve been using Subversion heavily lately to keep my live and development sites synchronized and its been a huge productivity booster.  I had to move some code over to a lite site for work this evening and it was such a pain, because I had to go through and figure out the files I needed to copy.  With svn, or even cvs, I could use tags and then an update and let the computer do the grunt work.  Instead it was a 15 minute task and i broke the site twice due to missing dependencies.

Then, I wanted to do a quick update to soccerblogs.net, and needed to pull in code from another project within the repository using the svn:externals property.  I set the property and committed it, then when I tried to update my local development copy, the svn client would fail.  I had the following lines in my apache error log:

[Wed Feb 14 00:20:32 2007] [error] [client 65.23.154.104] client denied by server configuration: &lt;PATH TO SVN&gt;

The svn client would complaing about "PROPFIND: 403 Unauthorized".  I triple cheked my apache+svn configuration, and I could browse my repository just fine in a web browser.  It turns out that if you have mod_evasive installed, the access pattern from svn can look like a denial-of-service attack to it, so it blocks with a 403.  Disabling mod_evasive did the trick – and I didn’t really need it anyway.  None of the English language google results mentioned this, but this one in Spanish had it.  Sometimes, being bilingual pays off.

Fail2Ban + Mod_Security = Spammer Bouncer

Under debian, fail2ban’s configuration is in /etc/fail2ban/.  In the filter.d directory add the following file and name it apache-modsec.conf.

[Definition]<br /># Option:&nbsp; failregex<br /># Notes.:&nbsp; regex to match the password failure messages in the logfile. The<br />#&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; host must be matched by a group named "host". The tag "&lt;HOST&gt;" can<br />#&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; be used for standard IP/hostname matching.<br /># Values:&nbsp; TEX<br />#<br />failregex = [[]client &lt;HOST&gt;[]] mod_security: Access denied with code 500<br /># Option:&nbsp; ignoreregex<br /># Notes.:&nbsp; regex to ignore. If this regex matches, the line is ignored.<br /># Values:&nbsp; TEXT<br />#<br />ignoreregex = 

Enable this filter by  adding the followin to jail.local:

[apache-modsec]<br />enabled = true<br />port&nbsp;&nbsp;&nbsp; = http<br />filter&nbsp; = apache-modsec<br />logpath = /var/log/apache*/*error.log<br />maxretry = 4

Restart fail2ban, next time a spammer gets blocked by modsecurity 4 times, it will be recorded in /var/log/fail2ban.log

2007-02-07 11:52:45,024 fail2ban.actions: WARNING [apache-modsec] Ban 207.234.131.237

Google Calendar Launches

Google Calendar has launched if you want to check it out. There is also an overview of the features available. Will this be a compelling calendaring solution? After looking it over and checking out the website, I think it might be for me. I’ve setup DAV on apache before so that I can have a portable read/write calendar available online but it hasn’t become critical for me. Of course, I was also the kind of person all through school and college that kept track of due dates pretty much in my head (and that’s why professors also give you a syllabus). This might just stick, at least I’ll try it out and report on if it does in the near future. This summer is full of travel and events so its as good a time as any.

Some observations on the funcitonality

  • Drag in grid to create events (ala outlook)
  • You can subscribe to any public ical/ics file (doesn’t handle stuff behind a password protection)
  • You can invite friends to view your calendar (even if it is not public)
  • You can search through other public calendars
  • You can invite people to events in your calendar, but I don’t see an option for scheduling the time/date conflicts
  • Event reminders via email or sms(to cell phone)
  • Can import a calendar from iCal or csv export from MS Outlook
  • Integrates with gmail
  • You can invite people via email to an event and they can reply if they will attend or not, ala evite but without the spam
  • You can subscribe to your calendar to view it via Mozilla’s Calendar app or Apple’s iCalendar.
  • Event organizers can embed a little button on their own pages to prompt users to add an event to the user’s own google calendar ()

Som things Google did not find essential, for this release and possible any other:

 

  • Scheduling free/busy for an event across all participants
  • Synchronizing calendars with other devices. Of course, if you have iCalendar on apple, iSync should take care of synchronizing with your cell phone or PDA.

 

Pointers: running your own mail server

There are a bunch of reasons why you might decide that running your own mail server is something you want to do. You have a lot of free time and enjoy spending a lot of time at the command line reading howto guides and installation manuals. Hopefully, you know you are running a mail server and don’t have a Windows machine that’s been turned into a spam zombie.

Ok, so that may not sound like good reasons. There are some real benefits, you can give yourself unlimited email aliases, give your friends and family easier-to-remember addresses, and set up mailing lists to keep in touch with people.

If you’ve got a linux server, Postfix is one of the more popular mail transports. One of the things about mail server jargon is that there are a number of lego blocks that go into the mail chain. I won’t attempt to write a complete guide to setting up your server, instead I’ll point you at some useful links that I found helpful, and to boot not horrendously diffult to install.

  • Postfix takes care of receiving incoming mail and routing it to a local destination. I found the Postfix Anti-UCE Cheat Sheet useful for making sure I had configured the server correctly to make sure it is being used for good and not evil.
  • Amavisd-new is a perl script for plugging in virus scanners and spam blocks into the delivery chain. I use Clam Anti-virus, a Free virus scanner, and Spamassassin, to protect users from unwanted or dangerous mail messages.
  • Once you’re system is running, you might find that you’re looking at the mail log to make sure nothing is out of sorts. Download pflogsumm and schedule it to send you a report on how many messages are being delivered/sent/blocked and other useful metrics.

PHP File Uploads

Yesterday at work I spent the better part of the afternoon trying to figure out why a form couldn’t handle multiple file uploads. These uploads where failry big – quicktime and wmv files – so I thought the culprit was the large file size. But the script was not returning any error warnings or notices which made it quite a pain to debug. With a co-workers help we traced it to two settings in php.ini.

  • post_max_size = (some number)M
  • upload_max_filesize = (some number)M

post_max_size controls the total size of your post request while upload_max_filesize controls the max size of each file uploaded. The sum of file sizes can’t exceed the maximum size of the post request.

A related apache setting could come in to play here, LimitRequestBody, which also sets a limit on how large a post request can be. So, in order to have working uploads for your php scripts, these 3 numbers have to jive. LimitRequestBody = post_max_size >>> upload_max_filesize