missing SOA records and PowerDNS sending ServFail

So today I was trying to resolve an issue with a domain we host. Two of our name servers were sending the correct info, the third was not. In the logs of the third name server I was seeing entries like this:

Mar 1 08:26:22 ns3 pdns[19262]: Not authoritative for <domain> sending servfail to <ip>

It turns out that this is caused by missing SOA records for the domain. In fact in my database there were NO records for this domain. I added the appropriate records in and it now serves!

Samba veto and .DS_Store OS X files

So i have a samba file share that is accessed by several macs here at work. and since I do not like their .DS_Store files, I have added a couple of lines to the smb.conf to make my live easier! In the [global] section, add these lines:

veto files = /.DS_Store/._.DS_Store/
delete veto files = yes

mysqldump and non-latin characters

So today we had an issue where transferring a MySQL database from a development server to a production server caused the text on some pages to have weird characters where apostrophes should have been. My initial thoughts of conversion between character sets was right on the money, but the fix was a little weird. You need to tell mysqldump to export in latin1 and mysql client to import in UTF8, for example:

 mysqldump -uUSER -pPASSWORD -hHOST \ 
--default-character-set=latin1 --skip-set-charset SCHEMA \
|mysql -hHOST -uUSER -pPASSWORD --default-character-set=utf8

iSCSI and windows 7 home premium backups

So my new laptop decided to let me know to that I should setup some backups to protect my data. I was somewhat disappointed when I was not able to backup to a network target so I went in search of an answer.

I found my answer in iSCSI. I setup my linux server as an iSCSI target and fired up the windows 7 iSCSI Initiator, connected it and then formatted my new disk. Windows backup was more than happy to write to a “local” disk for it’s backups.

So how did I do this? well I setup a 50gig lun1.img file and setup my iSCSI target according to this article on setting up iSCSI on ubuntu and then followed this article on technet for the windows side of things.

Problem Solved! This is the first time I have setup iSCSI and it was surprisingly simple

Handy Linux Tips: diff recursive compare on folders

Handy dandy tip to compare two project folders to see which files are different.

diff -rq --exclude='\.*' folder1 folder2

Quick Explanation:

  • -r is for recursive
  • -q (or –brief) is for just showing which files are different
  • –exclude=’\.*’ excludes all dot files and folders (great for excluding .svn and Zend frameworks .cache folders)
  • Folder 1 – the new one
  • Folder 2 – the original one

Handy grep command for svn

To quickly search through all the files in a project and not the .svn folders, here is a simple grep command to get you there.

grep -R --exclude-dir="*/.svn*" PATTERN *

SVN Protocols and Non standard SSH ports

Here is an interesting thing I learnt today, you can create customer SVN protocol handlers. Why on earth would you want to do that? Well sometimes you want to use a non standard ssh setup!

you can define your non standard ssh protocol quite easily in the [tunnels] section of your ~/.subversion/config file. for example:

[tunnels]
ssh1234 =  ssh -q -o ControlMaster=no -p1234

SSH pubkey fun

I had an interesting issue today where I was not able to use ssh key pairs to log into a box. It turns out that I had loose permissions on the home folder and ssh was silently failing.

So with that in mind, here is my guide to file system permissions for successful ssh authentication.

The home directory must not be writable for the world, but locking writes down to your user is probably a good thing too. the .ssh folder inside your home folder must be no looser than 0755 and the files inside .ssh *should* be 0600 to keep things nice and locked down tight.

chmod 755 /home/user
chmod 755 /home/user/.ssh
chmod 600 /home/user/.ssh/*

The home folder permissions is what tripped me up today, but a quick google and then chmod sorted that out for me.

Atlassian Jira/Crowd Troubleshooting

So Recently I had an issue with Atlassian Jira in which pages listing issues were not updating, but the issues themselves were.

Simple Resolution I thought as I removed backups from the previous year and freed up some disk space. I stopped Jira, with a

/etc/init.d/tomcat5.5 stop

Then removed the old backups and started jira again.

But that is when another problem arose – no one could log in! oh shite… how the hell do I fix this?!

It turns out that when you stop tomcat (and therefore Crowd)  Crowd writes out a config file, but since the disk was full, it could not write the file to disk!!! So, here is the file in question

ll ./atlassian/crowd/home/crowd.cfg.xml
-rw-r--r-- 1 tomcat55 nogroup 0 2011-06-15 10:42 ./atlassian/crowd/home/crowd.cfg.xml

See that 0 – that’s bad mkay. Luckily we have system backups and I skitched the file from there, stopped tomcat put the file in place and started it again.

we can now login – WOO fricken HOO… except where the heck are all of our issues?!

Here we go again. TM

The issues themselves are there but somehow the issues are no longer associated with their users.

This was easily enough resolved, I stopped tomcat, moved the old “jira.home” directory out of the way, created a new empty directory (assigning it the correct user permissions) and then restarted tomcat. This forced Jira to recreate a lot of things. After a few users browsed around in Jira for a bit the indexes were created again and all was good.

Side Note: The Opera web browser likes to cache things. If you are getting blank pages for your “Assigned to me” gadget, delete your cookies for your Jira installation, close your browser and start it again.

DNS Glue

So there is this wonderful thing called DNS Glue I learnt about the other day and it is used to break circular references in DNS entries.

What is a circular reference I hear you ask?

It is when a subdomain (such as ns1) is used to  tell there world where it’s parent is. The way DNS works when you ask for a domain is it starts at the TLD (say .com) and works its way down through the domain name. The first request is “where is .com”, the second is “Where is example.com” and the third and final is “where is ns1.example.com”. The problem here is ns1.example.com is responsible for telling the world where to find example.com.

This is where DNS Glue comes in,  It tells the DNS system that “if you are looking for ns1.example.com to look at this IP address here”.

To do this on planet domains is simple enough. Log into your account (if you are a whitelabel reseller, go to the user who has the domain and grab the login details and then go to your reseller site and log in). Once there, find the domain you want to add the DNS Glue to and click the plus symbol and add the domain and IP address to the “Child Name Servers” box and then you are golden.

I did have an issue with this though as I used the Planet Domain API to set this initially and it did not sync correctly with the registrar. From the reseller whitelabel admin I was able to use the “sync” button on the domains page to resync the planet domains database with the registrars version and then set it up again and it worked!

A couple of links that I found that were very helpful where