when rescan-scsi-bus.sh fails to notice disk size increases

Posted on by

So usually when I am doing online capacity expansions of vmware/raid devices I use a tool called “rescan-scsi-bus.sh” and it works. It detects the size increase on the disk and then I run a resize2fs /dev/sdb or something like that.
Well, I have come across a time when this tool does not work and I need to do things the hard way. Luckily the hard way is rather simple.

  1. Get the SCSI id of the device you want to use. The tool “lsscsi” does this nicely 
    # lsscsi
[1:0:0:0] cd/dvd NECVMWar VMware IDE CDR10 1.00 /dev/sr0
[2:0:0:0] disk VMware Virtual disk 1.0 /dev/sda
[2:0:1:0] disk VMware Virtual disk 1.0 /dev/sdb
  2. Now comes the hard part, you need to tell the server to rescan this device. 
    echo 1 > /sys/bus/scsi/devices/2\:0\:1\:0/rescan
  3. And now you watch dmesg for the expansion message! 
    dmesg | grep change
[4768364.446120] sdb: detected capacity change from 171798691840 to 268435456000
[4768364.834677] VFS: busy inodes on changed media or resized disk sdb
{lang: 'en-GB'}

wvproxy.com and the meaning of it all

Posted on by

So I was trawling through the log files here at work today and I noticed some more wvproxy.com URLs coming through to one of our customers. So I picked up the phone and called them.

It turns out that they have a Yellow Pages listing and have an alternate phone number to call on their yellow pages site. But strangely the number on the yellow pages site is NOT the same as the one that wvproxy.com returns.

The number on the wvproxy.com site does go through to the client though.

{lang: 'en-GB'}

MD Raid 5 with 2 dropped disks

Posted on by

So today I was happily going about stuff when there was a brown out. nothing turned off so I was happy to continue going about doing whatever I was doing. Until I got these emails

Subject: Fail event on /dev/md1:server
This is an automatically generated mail message from mdadm
running on server

A Fail event had been detected on md device /dev/md1.

It could be related to component device /dev/sdg.

Faithfully yours, etc.

P.S. The /proc/mdstat file currently contains the following:

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid5 sdi[3] sdh[1](F) sdg[0](F)
2930274304 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/1] [__U]

md0 : active raid5 sda[5] sdb[0] sdd[1] sde[3] sdc[4]
7814051840 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]

unused devices: <none>

and

Subject: Fail event on /dev/md1:server

This is an automatically generated mail message from mdadm
running on server

A Fail event had been detected on md device /dev/md1.

It could be related to component device /dev/sdh.

Faithfully yours, etc.

P.S. The /proc/mdstat file currently contains the following:

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid5 sdi[3] sdh[1](F) sdg[0](F)
2930274304 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/1] [__U]

md0 : active raid5 sda[5] sdb[0] sdd[1] sde[3] sdc[4]
7814051840 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]

unused devices: <none>

After the small panic attack that my backups raid array had gone walk went about trying to fix it.

Root Cause Analysis

So It turns out that the power brown out had knocked two of my disks offline for a brief second and when they came back online Ubuntu said “oh hey – new disks!” and promptly decided to give them new names. This in turn caused my /dev/md1 array to go bonkers.

The Fix

Well, I wanted this fixed quick – so I turned my Linux server off and then on again – thinking that the disks will renumber themselves back into what they should be and the MD1 array would reassemble.

I was half right, I still needed the array to assemble and run after the reboot. Here is the command I used to get things going again.

mdadm --assemble /dev/md1 --force --run

and there we are, making an MD array run properly after power failure. I am just lucky that there was no writing going on at that time.

Maybe time for a UPS and some ZFS loving?

{lang: 'en-GB'}

Proud Daddy Nerd Moment

Aside

So today my 4 year old wrote her first sentence in IRC.

[09:47:56] <Boxie> tiana was here.
[09:48:23] <Boxie> ^^ and that is Tiana’s first proper sentance written in IRC :D

{lang: 'en-GB'}

More wvproxy.com weirdness

Posted on by

So I just had a call from David from http://www.bigearsmediagroup.net.au/ doing an investigation into wvproxy.com (see my wvproxy.com weirdness post) and he had some very interesting things to say.

Apparently they are now doing some Google Adwords advertising and they are still changing the phone number and inserting extra stuff. This code was taken from the wvproxy version of the website: http://www.sdwetc.com.au/

<meta name="robots" content="noindex"/><meta name="norton-safeweb-site-verification" content="qelyedemzbc51m0-q79i41scmiq6-djrmmdl7v24ck9pcdy5gitxct9g32szorrblpfontb5rgctj1yviqav-ssxxu2w6y8xnncnl3kjx1tlssbboobwjbbie4igg10l" /><script language="javascript" type="text/javascript" src="http://prod-tct.webvisible.com/tracker-rp.js"></script>
<script language="javascript" type="text/javascript">
function wvEventLogger_defineSession() {
wvEventLogger = new Tracker({source: 'RP',
sessionId: 'wva0b72797751f4bd8beb9d882a1670282',
campaignId: '96083',
trackingHost: 'http://prod-tct.webvisible.com/'
});
}
function wvEventLogger_addLoadEvent(func) {
var oldonload = window.onload;
if (typeof window.onload != 'function') {
window.onload = func;
}
else {
window.onload = function() {
try {
oldonload();
}
catch (err) {
// ignore
}
func();
}
}
}

var wvEventLogger = null;
</script>

So as we can see, they are adding a noindex tag, so their goal is not to be indexed. They insert a norton safe search tag, although I do not know if this is legit or not. The change in phone number to me seems like a way to track conversions from the Google Adwords campaign – except the company in question does not have a google adwords campaign.

Could it be one of their competitors trying to steal their business?

At any rate – this is all rather – weird.

For the nerdy out there – a little bit of header checking led to an IIS6 Server behind an F5 load balancer web accelerator serving up the wvproxy.com domain

{lang: 'en-GB'}

Installing Dropbox on headless linux file server

Posted on by

So today I decided it was (finally?) time to get a dropbox account. The windows install went fine, but it appears that the linux version is geared up for GUI and not command line config. I had the following requirements:

  1. Put the dropbox folder in a place of my own choosing
  2. make it work without a GUI

so BEFORE installing dropbox, create a link to the place where you want files to be.

jason@server:~$ ln -s /data/documents/files/dropbox/ Dropbox

Then install dropbox.

sudo apt-get install nautilus-dropbox

And the first part was done, now I needed to hook up the install, as this was telling me

$ dropbox start
$ dropbox status
Waiting to be linked to an account...

so after some more digging, I found that I needed to start the dropboxd app manually and watch the output. so:

$ /var/lib/dropbox/.dropbox-dist/dropboxd
This client is not linked to any account...
Please visit https://www.dropbox.com/cli_link?host_id=19b...........................a7&cl=en_US to link this machine.

So, I went to that URL in the webbrowser I was already logged in with, it asked for my password – and then authenticated!

Client successfully linked, Welcome Jason!
{lang: 'en-GB'}

HTML5 input type=”number” and step granularity

Posted on by

or how I learnt the hard way about the how the step attribute works.

Being an old (VB4,5,6) windows developer, I mistook the “step” attribute on the <input type=”number”…/> HTML 5 element for what it used to be. It not only controls what the step is if you press the spin control but also defines what numbers are ALLOWED in the field.

For example, with the default step of “1″ you are only allowed to enter integers (that is whole numbers). With a step of 2 you are only allowed to enter even integers. With a step of 0.1, you are allowed to enter numbers with a single decimal place, 0.5 means only half integers and integers (…-0.5,0,0.5, 1.0, 1.5…).

So if you find yourself getting errors from your browser when trying to enter 41.2 into a number field, try setting step to either “0.1″ or “any”.

More reading over at HTML: The Markup Language on the W3C website.

{lang: 'en-GB'}

missing SOA records and PowerDNS sending ServFail

Posted on by

So today I was trying to resolve an issue with a domain we host. Two of our name servers were sending the correct info, the third was not. In the logs of the third name server I was seeing entries like this:

Mar 1 08:26:22 ns3 pdns[19262]: Not authoritative for <domain> sending servfail to <ip>

It turns out that this is caused by missing SOA records for the domain. In fact in my database there were NO records for this domain. I added the appropriate records in and it now serves!

{lang: 'en-GB'}

wpsc_update_custom_meta() nukes my custom post meta data

Posted on by

So today I was rather perplexed as to why my custom meta box on my post edit page was not playing ball. It would be set to a value and always stay at it, delete it from the database and give it a different value – and it would keep that too.St

Since I had used the update_post_meta() function before and it had never behaved this way, I looked through the mysql.log file on my server and saw this

4162 Query SELECT meta_id FROM wp_postmeta WHERE meta_key = 'my-plugin-key' AND post_id = 128
4162 Query UPDATE `wp_postmeta` SET `meta_value` = '0' WHERE `post_id` = 128 AND `meta_key` = 'my-plugin-key'
4162 Query SELECT meta_id FROM wp_postmeta WHERE meta_key = 'my-plugin-key' AND post_id = 128
4162 Query UPDATE `wp_postmeta` SET `meta_value` = '1' WHERE `post_id` = 128 AND `meta_key` = 'my-plugin-key'

What the? my value is set and then set back to what it was!? Time to find out where this was happening. Since I use Nusphere’s PhpEd to develop all my PHP projects, I fired up the trusty debugger and followed the code. Now take a guess where my setting is being undone!

Now I realise that I am playing in WP E-Commerce’s own playgound and that I need to play by their rules – but overwriting my changes – that’s just not cool!

The Fix

Go Last. Well, have wordpress call your save function after wp e-commerce. It’s rather simple actually – but a gotcha that cost me a couple of hours!

add_action( 'save_post', 'MyPluginHandleMetaBoxSaveData', 99);
{lang: 'en-GB'}