The jQuery team has put together a site devoted to the new version, called the 14 days of jQuery, covering the major version changes as well as infrastructure updates coinciding with the new release. For example, the documentation site has been completely redesigned, and been moved to it’s own subdomain home, api.jquery.com. Links from the primary jquery.com site should be updated within the next week. With video demos of new features, Q&A’s with the core team (including founder John Resig), it’s well-worth checking out for every jQuery developer.

 1) {
        preg_match("/^.+\.([a-z0-9\.\-]+\.[a-z]{2,4})$/", $host, $host);
        if (isset($host[1])) {
          return $host[1];
        } else {
          // not a valid domain
          return false;
        }
      } else {
        return $host;
      }
    break;
  }
}

// some examples
var_dump(get_top_domain('http://www.validurl.example.com/directory', 'all'));
var_dump(get_top_domain('http://www.validurl.example.com/directory', 'www'));
var_dump(get_top_domain('domain-string.example.com', 'all'));
var_dump(get_top_domain('domain-string.example.com/nowfails', 'all'));
var_dump(get_top_domain('finds the domain url.example.com', 'all'));
var_dump(get_top_domain('12.34.56.78', 'all'));
?>

Most of the examples are simply proofs, but I want to draw attention to the string in example #4, 'domain-string.example.com/nowfails'. This is not a valid URL, so the call to parse_url() fails, forcing the script to use the entire original string. In turn, the path part of the string causes the regex to break, causing a complete failout (return false;).

Is there a way to account for this? Surely, however I’m not about to tap that massive keg of exceptions (i.e. just a slash, slash plus path, slash plus another domain in a human-readable string, etc).

No regex for validating URL’s or email addresses is ever perfect; the “strict” RFC requirements are too damn broad. So I did what I always do: chose “what works” over “what’s technically right.” This one requires any 2-4 characters for a the top level domain (TLD), so it doesn’t allow for the .museum TLD, and doesn’t check to see if the provided TLD is actually valid. If you need to do further verification, that’s on you. Here’s the current full list of valid TLD’s provided by the IANA.

If you need to modify the regex at all, I highly recommend you read this article about email address regex first for two reasons:

1. There’s a ton of overlap between email and URL regex matching
2. It will point out all the gotcha’s in your “better” regex theory that you didn’t think about

If you haven’t already had a look, or missed a portion, here’s the full series:

1. Display Twitter updates on your website
2. Calculate dates and times in different timezones (translate Twitter timestamps)
3. Parse URL’s in text, create links (automatically link URL’s in stream)
4. Download and store your Twitter posts in a database

If you have any comments or questions, be sure to post them under the proper article.

Well no need for code-jitsu anymore. Based almost completely on a post I found at another blog — which is now down, how’s that for timing? — here are two functions which allow you to search for any occurrence of a specific token in a string…

/**
 * Find position of Nth $occurrence of $needle in $haystack
 * Starts from the beginning of the string
**/
function strpos_offset($needle, $haystack, $occurrence) {
  // explode the haystack
  $arr = explode($needle, $haystack);
  // check the needle is not out of bounds
  switch( $occurrence ) {
    case $occurrence == 0:
      return false;
    case $occurrence > max(array_keys($arr)):
      return false;
    default:
      return strlen(implode($needle, array_slice($arr, 0, $occurrence)));
  }
}

/**
 * Find position of Nth $occurrence of $needle in $haystack
 * Starts from the end of the string
**/
function strrpos_offset($needle, $haystack, $occurrence) {
  // explode the haystack
  $arr = array_reverse(explode($needle, $haystack));
  // check the needle is not out of bounds
  switch( $occurrence ) {
    case $occurrence == 0:
      return false;
    case $occurrence > max(array_keys($arr)):
      return false;
    default:
      $inverted = strlen(implode($needle, array_slice($arr, 0, $occurrence)));
      $actual = (strlen($haystack) - 1) - $inverted;
      return $actual;
  }
}

// look for second occurrence of letter 'a' from the start of string
echo strpos_offset('a', 'abracadabra', 2);
// returns 3

// look for second occurrence of letter 'a' from the end of string
echo strrpos_offset('a', 'abracadabra', 2);
// returns 7

In terms of use, we’ve essentially added an extra argument to strpos and strrpos that specifies which occurrence you’re looking for. In other words, you can make both of these functions work like the PHP standards by setting the third $occurrence variable to 1.

However, in the course of testing today, I realized that when a message is forwarded, sometimes the forward header will encode the email address, which gets stripped when I process the message. Allow me to demonstrate. Here’s the body an example message that someone might send to Fwd:Vault for safe keeping…

---------- Forwarded message ----------
From: "Office Flirt" <flirt@example.com>
Date: Wed, Jan 14, 2009 at 10:14 AM
Subject: Delete those images
To: you@example.com

My boss is sniffing around. I want you to delete those pictures I sent you right away.

Signed,
Office Flirt

Obviously you’re tucking this one away in Fwd:Vault to provide a little CYA-insurance when the boss calls you into his office. Good call. Now, before today, this message would come out of the scrubbing process looking like this:

---------- Forwarded message ----------
From: Office Flirt
Date: Wed, Jan 14, 2009 at 10:14 AM
Subject:
To: you@example.com
...

Look at the bolded red line. The email address is gone. You don’t have any other copies of it, so your boss doesn’t believe your story, and you get the blame. You’re forced to attend one of those god-awful sexual harassment classes. Fail.

So, what happened? Remember, you are looking at the body of a message in plain text. That “Forwarded message” block at the beginning is just part of the body text. So when the text was scrubbed by strip_tags(), the function picked it up as just another tag, which it dutifully removed.

To handle this situation, I came up with a piece of code that will look for email addresses in “tagged format” — i.e. surrounded by < and > — and remove the surrounding symbols, leaving us with harmless text.

$test = 'some surrounding text';
$test = preg_replace( "/(\<)(.+@[^\(\);:,<>]+\.[a-zA-Z]{2,4})(\>)/",
                      ' $2 ',
                      $test);
$test = preg_replace('/[\s]+/', '', $test);
echo $test;

Let’s break this down. First, we have a regular expression that identifies email addresses: .+@[^\(\);:,<>]+\.[a-zA-Z]{2,4}. This is the same expression set in the example on the Quanetic Software Regular Expression Tester (an excellent tool). We surround that in parentheses to isolate it as a subpattern. Then on either end of the expression, we tack on more regex voodoo to look for tag syntax: (\<) and (\>). These also get parentheses to identify them as subpatterns. Once its finished, we have an expression that will only match addresses wrapped in tagging structure.

The second argument in preg_replace() is the replacement, or what we should replace any matches with. In this case, we’ve isolated the address from the tags using subpatterns. So all we need to do is make a single call to the proper reference, which is $2, because its the second set of parentheses in the expression. Confused? You can learn about subpatterns on the PHP manual page for preg_replace().

Note the spaces around the $2 in the second argument. Sometimes the address will not have any spaces between the person’s name and the actual address. This could lead to the address being combined with the name which, in the case of Fwd:Vault, would screw up our search indexing. So we add spaces during the replace, then make a second call to preg_replace() to eliminate extra spaces: $test = preg_replace('/[\s]+/', '', $test);.

Legal Disclaimer: In case you do end up using Fwd:Vault when it launches, I’m fairly certain the service wouldn’t be liable in this silly hypothetical. Just make sure you read the terms before you sign up if you play the field at your office. Sorry to everyone going “duh” right now; it’s a sue-happy world.

Update: When I went to implement this change today, I discovered that the code was catching newlines (\n or \r) in the crossfire. It was actually due to the second call to preg_replace(), the “\s” character class includes not only spaces but line terminators as well. Oops. The revised version looks like this:

$body_text = preg_replace('/[ ]{2,}/', ' ', $body_text);

• Strings with letters
• Strings with numbers
• Strings with letters and numbers
• Uppercase, lowercase
• Fixed, variable length strings
• Option to include symbols

Problem is, none of them ever incorporated all this functionality. Every generator was a hodgepodge, e.g. some forced inclusion of numbers, or allowed either upper or lowercase, not both. All of these are great options, and it would be great to have all of them at your disposal in one tight function.

No more! I got so sick of finding shortcomings that I finally just put it all together myself. The following function allows you to choose a string length, as well as the character sets to use when building your random string. You can even include a set more than once, giving greater usage weight to certain characters. Finally, complete flexibility!

function koehl_generator($length = 10, $charsets = 'lower') {
  $upper = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
  $lower = 'abcdefghijklmnopqrstuvwxyz';
  $numbers = '1234567890';
  $symbols = '!@#$%^&*()-_=+<>,.?/:;[]{}|~';

  if (!is_array($charsets)) $charsets = array($charsets);
  $charset_pool = array();
  foreach ($charsets as $set) {
    $charset_pool[] = $$set;
  }
  $max = (sizeof($charset_pool) - 1);
  $v = '';
  for ($i = 0; $i < $length; $i++) {
    $this_pool = $charset_pool[rand(0, $max)];
    $v .= $this_pool[(rand() % strlen($this_pool))];
  }
  return $v;
}

// usage examples
echo koehl_generator(10, 'upper') . '';
echo koehl_generator(10, 'lower') . '';
echo koehl_generator(10, 'numbers') . '';
echo koehl_generator(10, 'symbols') . '';
echo koehl_generator(10, array('upper', 'lower')) . '';
// order of the array in the second argument does not matter
echo koehl_generator(15, array('lower', 'upper', 'numbers')) . '';
echo koehl_generator(15, array('lower', 'numbers', 'upper')) . '';
echo koehl_generator(15, array('upper', 'lower', 'numbers', 'symbols')) . '';
echo koehl_generator(15, array('lower', 'symbols')) . '';
// note how often letters appear in the next one
echo koehl_generator(20, array('lower', 'lower', 'numbers')) . '';

Adding your own custom set is easy too. Include a new set following the syntax of the existing ones, then call your new set by variable name in the second argument...

// add this to the top of the function...
$the_basics = 'abc123';

// then use it like so...
echo koehl_generator(5, 'the_basics') . '';
echo koehl_generator(10, array('the_basics', 'symbols')) . '';

If you find this useful, please be sure to give me some link love, just a reference URL to this page in your code would be fine. Using the share buttons below would be great as well!

So while searching the web, I came across an insightful post that discussed PHP’s built-in DateTime object. I try hard to keep up with all the tech in my job, but this one got through the cracks, and I was instantly intrigued. So after some research and fiddling, I came up with the following block of code, which will allow you to easily translate dates/times to and from any timezone.

date_default_timezone_set('GMT');
$datetime = new DateTime('2008-11-25 16:48:13');
$tz = new DateTimeZone('America/New_York');
$datetime->setTimezone($tz);
$time_display = $datetime->format('D, M jS g:ia T');

The process is a little convoluted, so here’s what’s going on. First we must set the system-wide timezone to match that of the date/time we would like to translate. In this example, it’s about 4:50pm on Nov 25, 2008 in the GMT timezone. Next we create a DateTime object with that time (FYI you can use any format accepted by strtotime()).

Next we create a DateTimeZone object. Keep in mind that this step stands completely on its own, and can be performed anytime up to now.

Finally, we pass the DateTimeZone object as the lone argument in a call to the setTimezone() method. This will change the timezone stored in $datetime from GMT to EST, and automatically update the date/time accordingly, which we output with a call to the format method.

Note that I used object-oriented syntax for this process, but there is a procedural style (i.e. functions) as well. I prefer OOP here because it’s just easier to read and follow.

Valid arguments for date_default_timezone_set() and DateTimeZone() come from PHP’s list of supported timezones.

So now that we can translate timezones, let’s apply it to my Twitter example…

status as $key => $status) {
      $datetime = new DateTime($status->created_at);
      $datetime->setTimezone($tz);
      $time_display = $datetime->format('D, M jS g:ia T');
?>
text . '' . $time_display; ?>


more...

Sidenote: While I was working on this I found GMT and UTC used almost interchangeably. For all intents and purposes, they are identical. But there is a technical difference: GMT is based on the rotation of the earth (less precise), while UTC is based on an atomic clock (more precise). Don’t worry, you and I are not the only ones who got confused.

Build a slick Twitter feed on your site

1. Display Twitter updates on your website
2. Calculate dates and times in different timezones (translate Twitter timestamps)
3. Parse URL’s in text, create links

So, if you need to get the timestamp of an RFC2822 e-mail header—or most any RFC-based timestamp—into MySQL DATETIME format, it’s easy:

$timestamp = strtotime('Tue, 30 Sep 2008 10:30:00 EDT');
echo date('Y-m-d g:i:s a', $timestamp);

Keep in mind that if there are differences between the timezones of your server and the timestamp, the date() function could screw with your desired output. You should probably take a look at date_default_timezone_set().

When I started searching around for resources on how to automate the SFTP connection, I was hit in the face with tons of dead ends. Several Google searches were spitting back mailing list and forum archives of plenty of questions regarding how to create backup scripts that connect to a remote server via SFTP. If you are in this boat, read on.

Here’s the problem. At some point in the development of SFTP, the writers decided that storing access credentials in files as part of an automated process was a very bad idea. So they coded SFTP to bypass the password challenge when invoked from a script (aka the -b flag, which runs commands from a file).

Instead, they recommend that you create a private key pair between the two systems. This preemptive measure handshake eliminates the need for passwords entirely, making your code a bit simpler. It’s fairly easy to do but, of course, most developers groan at the thought of having to learn yet another technique, and looks for ways around the restriction. I did both, and recommend the key pair approach. I’ll describe both here, and let you decide for yourself.

SSH/SFTP connection without passwords

The following example is borrowed from an article on The Linux Problem Base, but there are several out there explaining the same approach.

First log in on A as user a and generate a pair of authentication keys. Do not enter a passphrase:

a@A:~> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/a/.ssh/id_rsa):
Created directory '/home/a/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/a/.ssh/id_rsa.
Your public key has been saved in /home/a/.ssh/id_rsa.pub.
The key fingerprint is:
3e:4f:05:79:3a:9f:96:7c:3b:ad:e9:58:37:bc:37:e4 a@A

Now use ssh to create a directory ~/.ssh as user b on system B. The directory may already exist, which is fine:

a@A:~> ssh b@B mkdir -p .ssh
b@B's password:

Finally append a’s new public key to b@B:.ssh/authorized_keys and enter b’s password one last time. Note that you must be looking at the local directory in which you saved the key in step one.

cd /home/a/
a@A:~> cat .ssh/id_rsa.pub | ssh b@B 'cat >> .ssh/authorized_keys'
b@B's password:

From now on you can log into B as b from A as a without password. Try this to confirm:

a@A:~> ssh b@B hostname

It should return the hostname of system B without prompting for a password.

It’s painless and easy, and every SSH connection you make going forward requires less typing. However, if you still really want to use a password, you have two options.

Utilize the -o Flag

So the SFTP team made their stance clear, and backed it up with action. However, it’s not impossible to bypass the restriction. The -o flag allows you to access all the options available in the sshd_config file, so you can change any of them on the fly. Here we need to disable the batchmode directive, so your SFTP call would look something like this:

sftp -o "batchmode no" -b /tmp/bat user@host

I found this one on a random forum post, and it comes with an important warning:

Note that it must come *before* -b, which may be surprising – this is
due to ssh processing -o options as if they were read from the config
file – ssh_config(5) again:

The only problem here is that the password challenge gets sent back out to the command line, requiring normal keyboard interaction.

Use SSHPass

So if that’s still not good enough for you, check out a SourceForge project called SSHPass. From the link:

Sshpass is a tool for non-interactivly performing password authentication with SSH’s so called “interactive keyboard password authentication”. Most user should use SSH’s more secure public key authentiaction instead.

SSHPass is available from default Debian apt servers; I couldn’t find anything reliable on its availability through yum.

Proceed at your own risk. If you server allows any sort of public access, even to a large handful of outside users, I strongly recommend going the key route.

Update Feb 27, 2009: Reader pointed out that OpenSSH has a shortcut function, ssh-copy-id, to install your public key on a remote machine. Nice.

SELECT widgetID, count( widgetID ) AS totalWidgets
FROM table_widget_sales
WHERE totalWidgets > 1
GROUP BY widgetID
ORDER BY totalWidgets DESC

I want to see which widgets have been sold more than once, and how many times each of those remaining widgets have sold. Obviously we could pass a simpler query into a scripting language to calculate it for us, but that’s more cycles and memory. I love to squeeze as much out of my SQL statements as possible.

Problem is, when I run that query above, I get this error:

#1054 - Unknown column 'totalWidgets' in 'where clause'

My first reaction is, “I can use that column in the order by clause, why not where clauses!?” Simple, the value doesn’t exist when the WHERE clause is evaluated. The number is calculated as each matching record is passed. By the time we reach the ORDER BY clause, we have the data we need. But the WHERE clause is used to actually figure out what records we need to perform that calculation, so we’ve got a chicken-egg issue.

The solution is to use a subquery, sometimes called a “sub-select statement.” Using two select statements, we can split the duties for filtering (more than 1 widget sale) and counting (total number of sales per widget).

Here’s the working query:

SELECT w.widgetID, COUNT( ws.widgetID ) AS totalWidgets
FROM table_widgets AS w
INNER JOIN table_widget_sales AS ws ON ( w.widgetID = ws.widgetID )
WHERE (
SELECT COUNT( ws.widgetID ) AS number
FROM table_widget_sales AS ws
WHERE ws.widgetID = w.widgetID
) > 1
GROUP BY widgetID
ORDER BY totalWidgets DESC

See what’s going on? We’re doing the same count twice, but the subquery returns everything (number of sales per widget for all widgets), while the first select only grabs the results we want (sales greater than 1).