Archive for September, 2008

Format a RFC2822 date for mysql datetime fields

I was crash-coursing myself on PHP’s IMAP functionality recently, one of the first questions I came across was how I might store the date from an e-mail header in a MySQL DATETIME field. I was afraid I was going to have to parse out the string using a bunch of calls to substr(), but then I remembered that the strtotime() function recognizes various RFC formats.

So, if you need to get the timestamp of an RFC2822 e-mail header—or most any RFC-based timestamp—into MySQL DATETIME format, it’s easy:

[code="php"]$timestamp = strtotime('Tue, 30 Sep 2008 10:30:00 EDT');
echo date('Y-m-d g:i:s a', $timestamp);[/code]

Keep in mind that if there are differences between the timezones of your server and the timestamp, the date() function could screw with your desired output. You should probably take a look at date_default_timezone_set().


Automating SSH or SFTP in scripts

Recently I needed to automate copying a MySQL database to a backup server. We keep a copy of our site and DB on this box in the event that our main systems go down, or there’s a problem with our internet connection. It’s kind of like a poor man’s colocation setup. I actually prefer the setup over true colocation for the vast majority of small and medium-sized business, because it’s far simpler and requires far less overhead and continuous support.

When I started searching around for resources on how to automate the SFTP connection, I was hit in the face with tons of dead ends. Several Google searches were spitting back mailing list and forum archives of plenty of questions regarding how to create backup scripts that connect to a remote server via SFTP. If you are in this boat, read on.

Here’s the problem. At some point in the development of SFTP, the writers decided that storing access credentials in files as part of an automated process was a very bad idea. So they coded SFTP to bypass the password challenge when invoked from a script (aka the -b flag, which runs commands from a file).

Instead, they recommend that you create a private key pair between the two systems. This preemptive measure handshake eliminates the need for passwords entirely, making your code a bit simpler. It’s fairly easy to do but, of course, most developers groan at the thought of having to learn yet another technique, and looks for ways around the restriction. I did both, and recommend the key pair approach. I’ll describe both here, and let you decide for yourself.

SSH/SFTP connection without passwords

The following example is borrowed from an article on The Linux Problem Base, but there are several out there explaining the same approach.

First log in on A as user a and generate a pair of authentication keys. Do not enter a passphrase:

a@A:~> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/a/.ssh/id_rsa):
Created directory '/home/a/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/a/.ssh/id_rsa.
Your public key has been saved in /home/a/.ssh/id_rsa.pub.
The key fingerprint is:
3e:4f:05:79:3a:9f:96:7c:3b:ad:e9:58:37:bc:37:e4 a@A

Now use ssh to create a directory ~/.ssh as user b on system B. The directory may already exist, which is fine:

a@A:~> ssh b@B mkdir -p .ssh
b@B's password:

Finally append a’s new public key to b@B:.ssh/authorized_keys and enter b’s password one last time. Note that you must be looking at the local directory in which you saved the key in step one.

cd /home/a/
a@A:~> cat .ssh/id_rsa.pub | ssh b@B 'cat >> .ssh/authorized_keys'
b@B's password:

From now on you can log into B as b from A as a without password. Try this to confirm:

a@A:~> ssh b@B hostname

It should return the hostname of system B without prompting for a password.

It’s painless and easy, and every SSH connection you make going forward requires less typing. However, if you still really want to use a password, you have two options.

Utilize the -o Flag

So the SFTP team made their stance clear, and backed it up with action. However, it’s not impossible to bypass the restriction. The -o flag allows you to access all the options available in the sshd_config file, so you can change any of them on the fly. Here we need to disable the batchmode directive, so your SFTP call would look something like this:

sftp -o "batchmode no" -b /tmp/bat user@host

I found this one on a random forum post, and it comes with an important warning:

Note that it must come *before* -b, which may be surprising – this is
due to ssh processing -o options as if they were read from the config
file – ssh_config(5) again:

The only problem here is that the password challenge gets sent back out to the command line, requiring normal keyboard interaction.

Use SSHPass

So if that’s still not good enough for you, check out a SourceForge project called SSHPass. From the link:

Sshpass is a tool for non-interactivly performing password authentication with SSH’s so called “interactive keyboard password authentication”. Most user should use SSH’s more secure public key authentiaction instead.

SSHPass is available from default Debian apt servers; I couldn’t find anything reliable on its availability through yum.

Proceed at your own risk. If you server allows any sort of public access, even to a large handful of outside users, I strongly recommend going the key route.

Update Feb 27, 2009: Reader pointed out that OpenSSH has a shortcut function, ssh-copy-id, to install your public key on a remote machine. Nice.


Where the hell is Matt – feel inspired!

Deep down, I think everyone is inspired by the concept of world unity, but we usually leave it to air-headed beauty queens and Hollywood celebri-tards to talk about it. After all, if your not a world power or a big corporation, what power do you have to actually do anything? Here’s one creative answer…

This is apparently one of those Internet sensations and I’m simply late to the game. Oh well. I found it really inspirational and got a tremendous kick out of it. I hope it brightens your day as well.

You can find downloadable versions and more information at wherethehellismatt.com. Read more »


My Firefox extension collection

It seems to be some rite of passage for tech-focused blogs that they create a list of their favorite Firefox extensions. Who am I to break custom? If you’re not a web-developer, you can ignore the ones described as such, and consider the rest highly recommended. I have yet to find someone with as many extensions as me. You have been warned. Finally, you’ll find a screenshot demonstrating all these extensions in action at the bottom.


Adblock Plus with Adblock Filterset.G Updater

This is a fairly common standard. Adblock prevents ads from showing based on regex rules. I don’t have the time or inclination to write those rules, so I use Filterset.G to create them for me automatically. Between the two, I see no ads. Ever.

All-in-One Sidebar

It always drove me nuts that the download, extension, and theme panels popped up in their own window. This extension wrangles them into the sidebar area, just like the favorites. It includes an extra toolbar to toggle all the various sidebar areas, but can also contain any toolbar icon.

Better Gmail 2

Take Greasemonkey (which I don’t use), apply it to the Gmail interface, and you have Better Gmail 2. It adds some nice JavaScript-powered effects to the various pages, as well as additional information and links.

BugMeNot

I love using the BugMeNot service to circumvent those stupid sign-up requirements. I just want to read your article/download your software, get the hell out of the way! This plugin saves me from even having to visit the site. Right-click a login box, choose the BugMeNot option, and the login information is filled in automatically.

ColorfulTabs

Applies a different color to each tab, making each tab stand out a little bit better. Also includes options to associate the color with the content of the page.

ColorZilla

If you do any kind of design (but especially web design), you probably look at the web at least for some inspiration. This extension lets you grab any color displayed on any page. You can manually choose colors from the web-based palette, prepare a palette combination, and save your favorite colors.

Console2

Beefs up Firefox’s default error console. Search errors for specific text. Filter by JS, CSS, XML, chrome, or page content.

CustomizeGoogle

If Better Gmail 2 is the flash, this one is the bang. Goes deep into the top 16 Google properties and guts all the crap. Remove ads, remove sponsored links, force to secure connection (for things like Calendar and Docs).

Download Statusbar

Another generally popular pick. I use it for the same reason I use All-in-One Sidebar: it corrals Firefox’s bad habit of opening little extra windows. This one tucks downloads into a dynamic bar along the bottom. Each item is displayed as a progress bar, and includes a ton of download stats when moused over. After the download is done, double click to open/execute it, and the item is removed from the list automatically.

Extended Copy Menu

If you’re a developer, this is the must-have extension you’ve never heard of. It allows you to grab a copy of text from the browser in plain text format, i.e. without any special formatting. No more screwed up layout when you copy stuff into an e-mail or Word doc. Inversely, it also has a HTML copy option, which explicitly and cleanly grabs HTML along with the text.

Fancy Numbered Tabs

Replaces the closed graphic on tabs with a number. Like ColorfulTabs, I find it makes it just a little bit easier to navigate my tabs. It may bork the tab in some of the more elaborate themes.

Fasterfox

Another overall top pick. Fasterfox makes tweaks to Firefox’s configuration options (enter about:config into your address line to see them) to increase performance, sometimes drastically. Install, make sure it’s set to “Turbo Charged,” and you’re done.

Note: Fasterfox is currently marked as “Experimental,” which means you have to have an account on the Mozilla site to download it. I can say that it has worked fine for me, your results could vary (but probably not).

Firebug plus YSlow and Jiffy

At this point, this one really needs no introduction or explanation. If you do anything with the web, Firebug is a must-have. It has debugging options. Lots of them. JavaScript, CSS, XML, HTML, AJAX. This post is already too long, read the documentation for all the details.

YSlow and Jiffy are nice complements to Firebug’s standard functions. YSlow evaluates a given page and tells you how to speed things up. Includes links to the now-famous Exceptional Performance section of Yahoo’s Developer Network. Jiffy focuses on your Javascript and identifies which ones slowing up your page loads. Jiffy is also “Experimental,” like Fasterfox.

FireFTP

A solid FTP client is another required tool in any developer’s toolbox. I’ve tried all the usuals. CuteFTP, Filezilla, SmartFTP, WinSCP, each with their drawbacks. Lowsy interfaces, no reconnect, upload failure…Plus it’s another window I have to click, in addition to my browser and my coding interface. FireFTP has no downsides that I’ve found yet, sites right next to whatever site I’m working on, and is totally free.

Forecastfox

I don’t know anyone who doesn’t have this one. As if web-based weather wasn’t easy enough, now I don’t even need to browse anywhere to see the current, upcoming, and next day’s forecast. Mouseovers for radar and forecast details. Click any one for more information via AccuWeather.com (and with Adblock/Filterset.G, you won’t be bothered with their ads, either!). Automatic sliders for weather alerts.

Foxmarks Bookmark Synchronizer

Never worry about backing up your bookmarks again! Sign up for the free service (prompted automatically after installation), and Foxmarks will automatically maintain a remote copy of your bookmarks. If you’re away from your computer, you can access the bookmarks from their website. Side bonus: you can keep bookmarks in sync across multiple computers! I have an office laptop and a home desktop, and they always have the same bookmark list.

Gmail Notifier

There are tons of mail notifiers now, but if you use Gmail, this is one of the best. Easy setup, manage multiple accounts, automatic login, and a new mail alert slider.

IE Tab

Windows users have to go to the dark side for frequent security updates, and the occasional corporate site or intranet that Microsoft holds by the figs. Now you never have to leave the comforts of open-source home. This extension allows you to open a tab powered by the IE engine. Works with ActiveX plugins, so the Windows Updates site works fine.

Old Location Bar

I love the improvements in Firefox 3 save one: the auto-complete box that pops up as you type in the address bar. There’s so much going on in it, I find it impossible to discern the actual results at a glance. I don’t care about the page’s description, what’s the damn URL? Apparently I’m not alone. This extension brings back the simple listing behavior you saw in Firefox 2. Listed as “Experimental” like Firebug and Jiffy list above.

PDF Download

Ever click a link, only to get whacked in the face with that awful Adobe PDF plugin? Here’s an early warning system. When you click a PDF link, you are prompted with a series of options. Download it, open it using settings specified in the extension options, convert to HTML via PDF Download, use Firefox’s default behavior (usually aforementioned Adobe garbage), or cancel the download.

Bonus! Foxit PDF Reader

Okay, it’s not a Firefox extension, but I just can’t let one more person suffer under Adobe PDF Reader. That thing is a bloated, cpu-hogging, memory gobbling crap-fest. If all you need to do is view PDF documents (i.e. 99% of us), Foxit will meet your needs without running background clients (check your startup folder), and uses a fraction of the disk space.

Quick Restart

Ever notice that restart button you get when you install a new theme or extension? Now you can have that functionality whenever you want. Firefox still has a shifty memory leak, and this extension really saves me when it rears its ugly head. Hit the button, browser off, browser on. Tabs are saved.

ShowIP

Shows the IP Address of the site you are viewing in the bottom taskbar. A must for web guys, but security wonks and general tech enthusiasts will like this.

StumbleUpon

Ever notice your browsing habits get into a rut? You look at the same sites, or you friends keep sending similar YouTube videos of people acting like idiots. Break out! Install the plugin, create an account, set your desired topics, and click the button. SU will send you off to places previously unknown. Give your destinations a thumbs up or down, and SU will learn to tailor its offerings to you (a bit). StumbleUpon has been gaining more and more steam for its unique approach to browsing. However I saw the potential early on. Check out that “Member since” date!

Tiny Menu

If you’re a frequent flyer on the menu bar (File, Edit, View, etc. alogn the top), just go right past this. But if you’re a snobby tech elitist who has evolved past menus and into keyboard shortcuts, you’ll love Tiny menu. Condenses all the menu bar options under a single icon, allowing you to use the bar for other purposes, or eliminate it entirely. I stretch my address bar across the length, and give the search bar it’s own line beneath.

Update Notifier

Firefox does a good job of looking for updates routinely, but only does so at startup. So if you leave your browser open for long periods of time (or indefinitely, like me), you never get those notices. Enter this extension, which checks for updates at a routine interval (adjustable, 24 hours by default), and displays notifies you by highlighting the menu bar icon. The icon also allows you to run a manual check. Another click will automatically install the updates, and a third will restart the browser. All of these steps can be automated.

User Agent Switcher

Another must-have for web developers and designers here. It changes the user-agent string to whatever you want. In layman’s terms, you can masquerade as any other browser. Great for testign browser-specific stylesheets, as well as security settings.

Web Developer

Complements really well with Firebug, though there is a fair bit of overlap. I use it a lot to resize the browser for specific pixel dimensions (how does this page look at 800 x 600?), outline elements for layout debugs, reset my browser cache, find broken images, test my Javascript graceful degradation (or, often, lack thereof), or measure an elements actual size with the ruler. Plenty of other useful data outputs.


Here’s what all that looks like in action. It’s a bit of a departure from the default layout, but it works for me. Hopefully you’ll find something useful in that list as well.


Turn off AVG e-mail signature

I am a huge fan of AVG Anti-Virus Free Edition. It provides the same level of virus protection as the pay-for packages—don’t be fooled, the differences are bells and whistles—and because it’s trimmed down it eats up less resources. I always install it as part of a comprehensive approach using several free Windows security tools.

However, how many times have you seen this at the bottom of an e-mail?

No virus found in this incoming message.
Checked by AVG – http://www.avg.com
Version: 8.0.169 / Virus Database: 270.6.14/1647 – Release Date: 9/2/2008 6:02 AM

This is especially great in e-mail conversations, where I’ll see this same text stacked up five, ten, fifteen times at the bottom. It’s a tremendous waste of space and makes scanning an e-mail conversation difficult to say the least. I don’t need to know that you have anti-virus software installed. If I did, I wouldn’t have installed it on my own computer. Duh.

AVG does this by default, and they don’t make it obvious at install time how to disable it. It may have something to do with the shameless plug they put in there, but I could be wrong.

If you want to get rid of the stupid thing, here’s how to do it in version 8…

  1. Double-click the AVG icon in your taskbar to bring up the control panel window.
  2. Go the menu bar, choose Tools, then Advanced Settings…
  3. Choose E-mail Scanner from the left-hand menu
  4. In the right-hand pane, clear the checkbox labeled Certify e-mail.
  5. Click the OK button at the bottom