August 3, 2011
12:57 am
by: Bill

Find out what process is listening on a particular port on a Linux server

I was recently trying to debug an app that had gone rouge on a server because of a PHP upgrade. It was a web based application listening on a particular port and after the upgrade it just returned 500 server errors. A quick scan of the Apache logs, and even a deeper grep of them returned no matching errors so I realized, although the application was using PHP it was using another process to serve the pages. The urls of the application had the port 8443, so I knew whatever was serving it up was listening on that port.

Enter netstat

By default netstat tries to give you a human readable list of services. But with the -n flag you can see the numeric ports. You’ll also need to add -l (show listening ports) or -a (show all ports) since the listening ports are not included by default. -p will include the process id and the “program” that is listening which is what we really want. Grep is a nice way to narrow things down after that…

1
2
3
$ netstat -nlp | grep 8443
tcp        0      0 0.0.0.0:8443          0.0.0.0:*
           LISTEN      1234/lighttpd

Posted in Command Line, grep, Server Administration, Web Servers Tagged , , , , , , ,

July 26, 2011
5:48 pm
by: Bill

Find all .htaccess files with a particular php configuration setting

After some testing on a server with a strange setup I needed to find all of the .htaccess files that might contain a particular php configuration directive. The pattern was unique enough that I would have probably just done a recursive grep for it in any other circumstance. But by default “grep -R someSearchString *” would ignore dot files like .htaccess, and on a bigger server this could have grep looking through a lot of files when I knew I only wanted to search .htaccess files. “grep -R someSearchString .*” would be just bad since it would recurse into the ‘..’ directory, so you would end up searching your whole harddrive.

The solution? “find”

1
2
3
$ cd /home
$ sudo find ./ -iname ".htaccess" -exec grep -H some_php_config {} \;
./someuser/httpdocs/.htaccess: some_php_config some_value

Sudo runs the command as root, since I’m looking in everyones’ home directories. find is the command that is actually looking for the files. Then it passes the matches to grep. The -H flag on grep gives you the name of the files, without it you would just get the matching lines from the files, but no indication which files they were in.

Just find all .htaccess files

In some cases it’s just nice to be able to find all of the .htaccess files

1
2
3
4
$ sudo find ./ -iname ".htaccess" -exec echo {} \;
./someuser/httpdocs/config/.htaccess
./someuser/httpdocs/.htaccess
./someuser/httpdocs/.htaccess

Posted in Command Line, find, grep, Linux, Server Administration, Web Servers Tagged , , , , , ,

July 21, 2011
2:30 pm
by: Bill

List installed packages with yum

How do I find out which packages are already installed with yum.

The command you’re looking for is ‘yum list installed’ but you might want to combine it with grep or less because it’s most likely going to be a long list.

Look for php packages:

1
$ yum list installed | grep php

Just page the whole list:

1
$ yum list installed | less

Of course now that you know that, perhaps you want to know what packages are available to you:

1
$ yum list available | less

And of course you should check to see what needs to be updated on a regular basis:

1
yum list updates | less

Posted in Linux, Server Administration, Web Servers Tagged , , , , ,

July 21, 2011
1:28 pm
by: Bill

Case Sensitive Comparisons in MySQL Where Clause

How can I make MySQL treat my query as case sensitive?

As with all good tools there’s more than one way to get the job done. Assuming you have access to change the structure of the table you have to think about the way you want to access the data by default. Is this something like a password that will always be case sensitive? (if it actually is a password, we should talk about not storing your passwords in plain text sometime.) Or is this something like a name that usually would be case insensitive but for whatever reason you want to make a case sensitive comparison?

Change the data to case sensitive permanently

Your first option is to alter the column to be case sensitive. Or in MySQL terms change the collation (or character set) to a case sensitive or “Binary” setting. Assuming the column is UTF8 you could do something like…

1
2
3
ALTER TABLE `MyTable` CHANGE `MyColumn`
  `MyColumn` VARCHAR( 3 ) BINARY
  CHARACTER SET utf8 COLLATE utf8_general_ci NULL;

Of course there are a variety of reasons this might be something you don’t want to make system wide. Not the least of which is the fact that a change like that would require a pretty extensive regression test in an existing system to make sure you don’t break something that was counting on the case insensitivity of that column.

One time case sensitive comparison in MySQL

The alternative is to just convert the column to binary inline. This will allow you to treat the row (or in the case of this example, the string) as case sensitive without altering the table structure.

1
2
3
4
5
6
7
8
9
10
mysql> SELECT
    ->     'a' = 'A' AS NormalCIEqual,
    ->     'a' = BINARY 'A' AS BinaryCS,
    ->     'a' = BINARY 'a' AS BinarySame;
+---------------+----------+------------+
| NormalCIEqual | BinaryCS | BinarySame |
+---------------+----------+------------+
|             1 |        0 |          1 |
+---------------+----------+------------+
1 ROW IN SET (0.00 sec)

This will also work with other types of comparisons…

1
2
3
4
5
6
7
8
9
mysql> SELECT
    ->     'a' LIKE 'A' AS NormalCILike,
    ->     'a' LIKE BINARY 'A' AS BinaryCS,
    ->     'a' LIKE BINARY 'a' AS BinarySame;

mysql> SELECT
    ->     'a' REGEXP 'A' AS NormalCIRegex,
    ->     'a' REGEXP BINARY 'A' AS BinaryCS,
    ->     'a' REGEXP BINARY 'a' AS BinarySame;

Posted in Code Snippets, Database, MySQL, SQL Tagged , ,

July 21, 2011
1:09 am
by: Bill

Cannot load Zend Extension Manager – it was built with configuration 1.2.0, whereas running engine is API220090626,NTS

After upgrading to PHP 5.3.x I’m receiving an error

1
2
3
4
5
$ php -v
Cannot load Zend Extension Manager
    - it was built with configuration 1.2.0,
    whereas running engine is API220090626,NTS
   ...

This is because you have Zend Optimizer installed which only works with PHP up to 5.2.x. You’ll want to comment out the Zend Optimizer configuration in your php.ini and then look into Zend Guard if you still need to work with “Zend Binary” code. Of course this presents another problem as scripts compiled with Zend Optimizer won’t work with Zend Guard. If you’re using third party “optimized” scripts and there is no Zend Guard version available or you want to continue to use Optimizer for other reasons you’ll probably want to move back to PHP 5.2.x.

Disabling Zend Optimizer

For me that meant commenting out the following lines in my php.ini

1
2
3
4
5
6
[Zend]
zend_extension_manager.optimizer=/usr/local/Zend/lib/Optimizer-3.3.9
zend_extension_manager.optimizer_ts=/usr/local/Zend/lib/Optimizer_TS-3.3.9
zend_optimizer.version=3.3.9
zend_extension=/usr/local/Zend/lib/ZendExtensionManager.so
zend_extension_ts=/usr/local/Zend/lib/ZendExtensionManager_TS.so

Posted in languages, PHP, Server Administration, Web Servers Tagged , , , , , ,

July 20, 2011
9:58 pm
by: Bill

How do I Tell What Version of Linux I’m Running?

How do I tell which version of Linux is on my server?

There are a number of ways, there is a command line tool `lsb_release` used to check the distribution, and version

1
2
3
4
5
6
$ lsb_release -a
LSB Version:    :core-x.x-ia32:core-x.x-noarch:graphics-x.x-ia32
Distributor ID: Fedora
Description:    Fedora release 14 (Laughlin)
Release:    14
Codename:   Laughlin

Unfortunately it is not available on all systems, especially if your system is really out of date. But you can try the following…

1
2
$ cat /etc/*-release
CentOS release 5.4 (Final)

Or…

1
2
$ cat /etc/*-version
Slackware 11.0.0

Posted in Linux, Server Administration, Web Servers Tagged , , , , ,

July 7, 2011
10:33 pm
by: Bill

July 7, 2011
9:35 pm
by: Bill

How do I Fix a Corrupt MySQL Table?

mysqldump Can’t open file: ‘tablename.MYD’. (errno: 145)

This usually means the table has become corrupt. Most commonly this happens when the partition that contains your MySQL files is full, but I have had it happen at other times. Especially on tables that get a lot of access like web sessions that get created and deleted often.

Now before you start kicking yourself because the last time you backed up was three years ago when you had your last strange crash you can try executing the following SQL command…

1
REPAIR TABLE MyTableName

That usually does wonders. Of course you will may need to resolve the underlying issues first, delete some files if your drive is full.

If it works for you (or even if it doesn’t) go set up a cron job right now to automatically back up your database regularly. Believe me, the piece of mind you will get from that when a problem arises is more than worth the time to set it up and the space to store it.

Another thing you might want to think about at a time like this is your storage engine. Yeah, I know, just what you want to think about. But the thing is, the default engine for older versions of MySQL “MyISAM” was screaming fast but the trade-off for that speed was less fault tolerance than might be expected from a modern DB. There are other options you might want to choose especially if you don’t have a UPS on your server, but you should be aware of the tradeoffs before you make the change. I’ll talk more about this in the near future.

Posted in Database, MySQL, Server Administration Tagged , , , , ,

July 2, 2011
6:26 pm
by: Bill

How to Clean Up a Mac File in VI with ^M Line Endings

Replacing ^M with new line in VI

Have you ever edited a file in VI that was created with a Mac text editor only to find you have one very long line with ^M characters where you are expecting line endings? Historically Mac OS used Carriage Return characters to indicate the end of a line. Most parts of OSX are more UNIX-like these days but there are still many tools that write text files with CR line endings. Those line endings display in a normally configured VIM installation as ^M

I’ve seen a number of strange search and replace solutions floating around, basically involving searching for a strange string of control characters. But there is a better way.

Permanently Changing the Line Endings

VIM (which is what you are running on most modern *nix systems when you type vi these days.) is able to convert the file for you very easily. Once you have a file open that you would like to change type the following commands.

1
2
3
:set ff=mac
:w
:e

The first line changes the “file format” to mac. The second saves the file. The third reloads the file and it should look clean. If you want to see what type of file vim thinks the file is you can type “:set ff?”.

1
2
:set ff?
fileformat=unix

Of course it’s a unix file now, but it thought it was a unix file to start with. (to confirm that you can load another unaltered Mac file and type “:set ff?” to see.) And therein lies the problem. Maybe it would be better to get VIM to understand mac files.

Configuring VIM to speak Mac

If you’re dealing with a lot of files created on a Mac it might be better to just reconfigure VIM. Which is not as hard as it may sound. The setting we want to alter is ffs, which is what tells VIM what kind of line endings it should be looking for. To see what your setting currently is type “:set ffs?”.

1
2
:set ffs?
fileformats=unix,dos

If you already have the file open and you want to reload it as a mac file WITHOUT actually changing the file you can do this…

1
2
:set ffs=unix,dos,mac
:e

Now you can edit and save the file without changing the line endings, which will probably be more convient for the original editor if they still need to use a tool that would prefer the original line endings. The first item in the file formats list is your default format. So if you want new files saved as mac, you can put it first in the list. When a file is loaded, the order makes no difference.

Of course that change only affects the current buffer. So unless you want to do that every time you probably want to edit your .vimrc file in your home directory. You don’t need the colon in the config file. From the command line just type…

1
echo 'set ffs=unix,dos,mac' >> ~/.vimrc

Now you should not have to worry about the dreaded ^M any more!

More VIM file format details can be found at http://vim.wikia.com/wiki/File_format

Posted in Command Line, Linux Tagged , , , , , ,

July 1, 2011
3:59 pm
by: Bill

How do I find out what my IP address is?

Or what does my browser look like to the outside world?

One of the problems with various kinds of network configurations and security is it might be a little hard to judge how your surfing looks to computers on the outside of your network. There are a number of reasons you might want to know for sure, the most common one I run into is debugging. (That and some of my upcoming posts will require that you know how you look to the outside world.) So here you are, your IP address, the way your browser is identifying itself, and some other interesting information your browsers sends via the HTTP Headers so the server knows how best to respond.

Your IP Address

38.107.179.226

Your browser’s “HTTP_USER_AGENT” string

CCBot/1.0 (http//www.commoncrawl.org/bot.html)

And a few other header variables for your entertainment

www.hellersoftware.com
HTTP_HOST
The domain name of the site your browser is requesting. Once upon a time, when HTTP/1.0 was the standard, and dinosaurs roamed free, web sites were only identified by IP address and port (usually port 80). Once the domain name was translated to an IP address, there was no real use for it. But IPV4 addresses are not infinite. As the internet entered the popular culture and everyone HAD to have a website it quickly became apparent that we were going to run out of addresses. The solution was to transmit the hostname along with the request. That allows the server to host as many sites at the same IP as resources will allow while still identifying the site you are requesting.
close
HTTP_CONNECTION
Tells the server if your browser would like to “close” the connection or “keep-alive” assuming there will be further communications with the server. Keep-alive is what you will see most of the time, since each image and support file is a completely separate request. Even if you view only one page, that page will usually consist of multiple HTTP requests.
text/html,application/xhtml+xml,text/xml;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
HTTP_ACCEPT
These are the types of files your browser will accept. Rather than depend upon file extension, the browser uses something called content type (historically know as MIME type for Multipurpose Internet Mail Extensions.) which has much more flexibility than a simple file extension.
gzip
HTTP_ACCEPT_ENCODING
The types of compression or other data encoding your browser will accept. This allows content to be served in a compressed format to save bandwidth.
ISO-8859-1,utf-8;q0.7,;q0.7
HTTP_ACCEPT_CHARSET
The text encodings your browser is looking for. We don’t all speak english, and some of those Kanji characters are very beautiful but they take a different type of character set to render them. These are the character sets your browser accepts.

Posted in Debugging Tagged , , , ,