History of demandred (shack server)

15+ Years of busted hardware and counting...

Entries by date:

1997

I had originally built a Linux server out of some spare parts to mess around with. It was left in my basement and it served no one but me. It had no direct Internet access and vice-versa. I started using it as a common access point for the computers in my home. It handled DHCP early on, later masqueraded a 28.8k modem and so on. This was the first time the name 'demandred' was used by me for a system. This was also the first time I used RedHat per one of my professor's recommendations. Up to this time I was running Slackware on my desktop.

Specs were:

IBM 486SLC-2/66
10MB RAM
512MB Samsung Hard Drive
10base2 NIC (ne2000-based)
Boca 14.4 modem
4x Mitsumi CD-ROM
Oak Video 256k VGA Card
RedHat 3.0.3 (and later RedHat 4.2)

July 2000

I had moved into a new apartment, my first apartment. There was no broadband available in my area so I put together a Windows 98SE machine to masquerade a 56k connection courtesy of NetZero. I couldn't rationalize paying for dial-up and this was temporary.

The specs are as follows:

Pentium 166
48 Mb RAM
768Mb Quantum Hard drive
10baseT Realtek 8029AS NIC
US Robotics 56k Modem
S3 Virge 4Mb PCI Video card
4x Mitsumi CD-ROM 
Windows 98SE

January 2001

DSL has arrived. Demandred is truly born. The server has mostly all new hardware, running RedHat 6.2. I had running my web server and mail server. Nothing really public at this point. However, it was my router for my DSL connection.
AMD K6/300
96Mb RAM
1.6Gb Western Digital HD
2x 100BaseT Realtek 8039
S3 Virge 4Mb PCI Video Card
4x Mitsumi CD-ROM
RedHat 6.2

May 2002

Demandred was a happy little box. By this time my RedHat 6.2 became a hybrid with EnGarde Secure Linux. I added the shack mailing list, some personal mail accounts for friends, added the on-line gallery, a decent backup scheme and a few more services. Then the WD 1.6Gb drive ate it. I swapped in a WD 2.5Gb drive and got the system running.
AMD K6/300
96Mb RAM 
2.5Gb Western Digital HD
2x 100BaseT Realtek 8039
S3 Virge 4Mb PCI Video Card
4x Mitsumi CD-ROM
RedHat 6.2 / EnGarde Secure Linux

November 2002

Demandred refuses to boot. "Li" is all I get. After numerous attempts to correct this I COULD not get the system to boot to the WD 2.5Gb drive. I moved the drive to share the secondary IDE with the CD-ROM and dropped an old 80Mb Seagate in there making it /boot and the boot drive. Voila, it worked. Also around this time power failures were pissing me off, I bought an IBM UPS.
AMD K6/300
96Mb RAM 
2.5Gb Western Digital HD (/)
80Mb Seagate (/boot)
2x 100BaseT Realtek 8039
S3 Virge 4Mb PCI Video Card
4x Mitsumi CD-ROM
IBM UPS 
RedHat 6.2 / EnGarde Secure Linux

January 2003

Demandred's K6/300 was showing problems. After 40 days of uptime she would oops. The oopses appeared to be hardware related. I dropped my old Pentium 150 in place of the K6/300 for the time being while hardware was tested.
Pentium 150
64Mb RAM
2.5Gb Western Digital HD (/)
80Mb Seagate (/boot)
2x 100BaseT Realtek 8039
Oak VGA 256k ISA video card
4x Mitsumi CD-ROM
IBM UPS
RedHat 6.2 / EnGarde Secure Linux

April 2003

The K6/300 was showing signs of serious problems when put under stress during testing. Memory checked out good. This hardware is no good for a server. Working with Chris Linstid he hooked me up with a great deal on a motherboard and chip for a K6-3/450. I used an old ATX case my parents' had laying around at their house and installed 10Gb Maxtor drive for /home, as with mail and the gallery the 2.5Gb was filling up.
K6-3/450
196Mb RAM
10Gb 7200RPM Maxtor HD (/home)
2.5Gb Western Digital HD (/)
80Mb Seagate (/boot)
2x 100BaseT Realtek 8039
S3Virge 4Mg PCI Video
4x Mitsumi CD-ROM
IBM UPS
RedHat 6.2 / EnGarde Secure Linux

September 2003

The 10Gb Maxtor failed! I was seriously pissed. This drive was fairly new and I screwed up something in my back up scripts and no backups were done for the month before this. I lost lots. I recovered some of what was on the 10Gb, trimmed things down to fit back on the 2.5Gb. Two hard drive failures and this odd boot problem with the 2.5Gb drive. No more screwing around with cheap, no good IDE disks. I bought 3 refurbished Seagate 18.2Gb Barracudas and a Adaptec AIC-7881U to run them from. I used two of the disks running software RAID 1 and kept the 3rd as a cold spare. This array became the new /home.

Pics and details can be found here regarding that system.

K6-3/450
196Mb RAM
2x Seagate Barracuda 18.2Gb (md0 -- /home)
2.5Gb Western Digital HD (/)
80Mb Seagate (/boot)
2x 100BaseT Realtek 8039
Adaptec AIC-7881U
S3Virge 4Mb PCI Video  
4x Mitsumi CD-ROM
IBM UPS
RedHat 6.2 / EnGarde Secure Linux

November 2003

Lost power, server won't boot. 80Mb drive DEAD. Oh well, no big loss, it was old. I replaced it with the 768Mb that was originally in the server from day one. 40Mb /boot and I used the rest for swap.
K6-3/450
196Mb RAM
2x Seagate Barracuda 18.2Gb (md0 -- /home)
2.5Gb Western Digital HD (/)
768Mb Maxtor (/boot)
2x 100BaseT Realtek 8039
Adaptec AIC-7881U
S3Virge 4Mb PCI Video
4x Mitsumi CD-ROM
IBM UPS
RedHat 6.2 / EnGarde Secure Linux

January 2004

Things were humming along nicely... then they stopped humming and started buzzing and stinking bad. Power supply blew. Back to the only reliable machine I had handy, the Pentium 150.
Pentium 150
64Mb RAM
2x Seagate Barracuda 18.2Gb (md0 -- /home)
2.5Gb Western Digital HD (/)
768Mb Maxtor (/boot)
2x 100BaseT Realtek 8039
Adaptec AIC-7881U
S3Virge 4Mb PCI Video
4x Mitsumi CD-ROM
IBM UPS
RedHat 6.2 / EnGarde Secure Linux

February 2004

I had the P150 in place so I tried a new power supply I got from Brent, but it was dead. At this point I was loosing patience. I have a dislike for ATX, so instead of buying a new ATX power supply I bought a new ATX case for $25 from MWave (which arrived damaged). I decided to use that case to run the K6-3/450 which will be swapped with a spare K6-2/450 I was using that was in an AT full-tower. I made the swap. After this I found that only 32Mb of 196Mb of RAM was good, damaged when the power supply failed. I have only 32Mb of ram to work with, I have ordered additional RAM. Also, I lost the 768Mb drive in the failure.

Brent hooked me up with 3 4.1Gb WD SCSI disks. I decided to use these. All IDE discs are removed from the system. I added 8 extra cooling fans to this case as well. Mounted all the drives in 5.25" bays with two sets of 3 cooling fans.

The final hardware specs are as follows:

K6-2/450
32Mb RAM (I will be increasing this soon)
2x Western Digital 4.1Gb SCSI discs (md1 -- /)
2x Seagate Barracuda 18.2Gb discs (md0 -- /home)
2x 100BaseT Realtek 8139 (eth1 is dropping packets)
Adaptec AIC-7881U
NVidia TNT2 16Mb AGP Video (overheats when you use it for games)
6x Sony CD-ROM drive
AT Full-tower 
IBM UPS
RedHat 6.2 / EnGarde Secure Linux

February 2004

More RAM :) I also upgraded the OS finally. I went from the RedHat 6.2/EnGarde Secure Linux hybrid to a full blown EnGarde Secure Linux Professional 1.5.4. I did the upgrade manually and hit a lot of snags. But I finally setup my own DNS and added a CVS repository at this time :)

Unfortunately the K6-3/450 is DEAD. From the power supply failure earlier. Built a new system with it, powered it on and all I got was a puff of smoke, a few blown capacitors and a few new words for my vocabulary.
K6-2/450
128Mb RAM
2x Western Digital 4.1Gb SCSI discs (md1 -- /)
2x Seagate Barracuda 18.2Gb discs (md0 -- /home)
2x 100BaseT Realtek 8139 (eth1 is dropping packets)
Adaptec AIC-7881U
NVidia TNT2 16Mb AGP Video (overheats when you use it for games)
6x Sony CD-ROM drive
AT Full-tower 
IBM UPS
EnGarde Secure Professional 1.5.4

To this date (Feb 18 2004) the following hardware has failed:

I would have had some nice hardware from my desktops that would have trickled down to this machine had I not had the following hardware fail in those as well: I hate computers.

April 2004

One of the 4GB SCA SCSI drives in the RAID 1 array failed. I have to replace it. Fucking wonderful.
md1 : active raid1 sdd1[1](F) sdc1[0]
3855488 blocks [2/1] [U_]

I/O error: dev 08:31, sector 3708088
raid1: Disk failure on sdd1, disabling device. 
Operation continuing on 1 devices

July 2004

Ok, so I was asking for trouble, luckily it only came in a small dose. I replaced the failed hard drive. The delay was due to a new job, moving to a new place and everything else involved so this failed hard drive too low priority for a while. So I replaced the drive and rebuilt the raid array without issue. However, the house got hit by lightning. I had the server on a surge protector and got lucky. However, the ethernet was not on a surge protector. And the phone lines carried the surge, through the DSL and through the ethernet network, I lost every ethernet card and router I had.

Recently I installed a Netgear Router to replace this machine as a firewall/router. This is the first time in Demandred's life that it has not hosted my Internet connection. So when the rtl8139 blew from the surge I had a spare one in this machine, not connected. So I removed the blown one and just hooked things up to the other one. Everything is running smoothly for now...
K6-2/450
192Mb RAM
2x Western Digital 4.1Gb SCSI discs (md1 -- /)
2x Seagate Barracuda 18.2Gb discs (md0 -- /home)
100BaseT Realtek 8139
Adaptec AIC-7881U
NVidia TNT2 16Mb AGP Video (overheats when you use it for games)
6x Sony CD-ROM drive
AT Full-tower 
IBM UPS
EnGarde Secure Professional 1.5.4

December 2004

Another hard drive failed. Fuck it, I don't even care to discuss it.

August 2005

Another hard drive failed. I had two Western Digital 4Gb SCSI drives in a RAID array. /dev/scd failed back in December. I neglected it and left it in there. I knew the day would come and /dev/sdd failed. It seemed mostly like a boatload of bad sectors started popping up. I have a backup routine running nightly. I managed to grab an 8Gb IDE disk from a friend's old computer, set that up with the demandred root file system on it and drop it right in. It actually went EXTREMELY smoothly so something is going to break soon. Thing NEVER go smoothly for me with this fscking machine. I lost only a couple e-mails in the process. This single IDE disk is only temporary, I will have a new RAID array in there shortly. I hope.

Oddly, note that exactly one year ago one of the 4GB SCSI drives died.

All-in-all I must say I feel blessed. Things ran for 8 WHOLE months without a problem before this. Then, when I am away on vacation, trying to upload some pictures one evening from my hotel the system eats it. After 8 months I take a vacation for 5 fucking days and it dies! I HATE YOU!!!

/me searches for a large, heavy, long, blunt object for beating the shit out of demandred with.
K6-2/450
192Mb RAM
2x Western Digital 4.1Gb SCSI discs (md1 -- /mnt/tmp) (both drives bad but the array is still "up")
2x Seagate Barracuda 18.2Gb discs (md0 -- /home)
Maxtor 8Gb IDE (/dev/hda1 /)
100BaseT Realtek 8139
Adaptec AIC-7881U
NVidia TNT2 16Mb AGP Video (overheats when you use it for games)
6x Sony CD-ROM drive
AT Full-tower 
IBM UPS
EnGarde Secure Professional 1.5.

And the RAID arrays:

Personalities : [raid1] 
read_ahead 1024 sectors
md0 : active raid1 sdb1[1] sda1[0]
17775808 blocks [2/2] [UU]

md1 : active raid1 sdd1[1]
3855488 blocks [2/1] [_U]

and of course, the wonderful hard drive failure errors in dmesg

scsi0: ERROR on channel 0, id 6, lun 0, CDB: 0x28 00 00 00 00 bf 00 00 08 00 
Info fld=0xbf, Current sd08:31: sns = f0  3
ASC=11 ASCQ= 1
Raw sense data:0xf0 0x00 0x03 0x00 0x00 0x00 0xbf 0x18 0xff 0xff 0xff 0xff 0x11
0x01 0x00 0x80 0x00 0xac 0x00 0x00 0x07 0x03 0x00 0x4a 0x00 0x00 0x00 0x00 0x00
0x00 0x00 0x00 
I/O error: dev 08:31, sector 128
raid1: sdd1: rescheduling block 128
raid1: sdd1: unrecoverable I/O read error for block 128
EXT2-fs error (device md(9,1)): ext2_read_inode: unable to read inode block - inode=416, block=16

March 2006

Anticipation builds, no hardware problems since the last one, except on my desktop systems, a CD-ROM drive ate it, an 80gb hard drive is dying and bad sectors are popping up on my laptop. But that's for another log.

the "anticipation" is my new server. Since my last message I have gotten married and moved (again). demandred, however, remains at my parents' place sitting in the basement. I don't plan on bringing it over. My current desktop, Athlon TBird 800 w/ 1.5Gb RAM will replace it shortly :) I hope...
Anticipated stats:
Athlon Tbird 800
1.5GB RAM
2x Seagate Barracuda 18.2Gb discs (md0 -- /home)
Unknown '/' disks
Intel Etherexpress 1Gb NIC
Adaptec AIC-7881U
ATI Rage 128 Pro 16Mb AGP
12/4/32x SCSI PlexWriter
ATX Full-tower
IBM UPS
Unknown OS (Dunno if I will keep EnGarde or move to Debian perhaps, or even try a BSD out)

June 2006

Okay, scratch the new server idea. My ex-wife and I bought our first home :) Demandred remains in my parents' basement. I have the SCSI RAID array pushing 3 years of age now... The '/' drive is a single 8GB Maxtor I snagged from a used Dell... I guess that's some kind of obvious foreshadowing for a soon-to-be entry in this list containing lots of swear words and ranting... It has also occurred to me, this is sort of a blog now... When I started this the term "blog" was almost unheard of :)

I still haven't removed the 4gb drives from demandred, they are still sitting there. The raid array is still "up", just "failed". Maybe when we move in to our new place I will bring demandred on over.

In the mean time, I lost another 80gb hard drive, bought a new flat panel monitor, upgraded the ram in my "gaming rig" to 1.5GB to match my other system and got a new, nothing special video card so I can play with some 3D stuff on my Linux box now :)
K6-2/450
192Mb RAM
2x Western Digital 4.1Gb SCSI discs (md1 -- /mnt/tmp) (both drives bad but the array is still "up")
2x Seagate Barracuda 18.2Gb discs (md0 -- /home)
Maxtor 8Gb IDE (/dev/hda1 /)
100BaseT Realtek 8139
Adaptec AIC-7881U
NVidia TNT2 16Mb AGP Video (overheats when you use it for games)
6x Sony CD-ROM drive
AT Full-tower 
IBM UPS
EnGarde Secure Professional 1.5.

June 2007

The server has finally been moved over to my house. I received a minor upgrade to one of my desktops so Demandred is getting some new hardware. Most notably upgrade to an 800Mhz Athlon (tbird) and 1.5GB of RAM. My ex-wife was pregnant with our first child and things have been getting busy. My desktop is running most of Demandred's services for the time and I haven't had time to rebuild demandred. I bought two more SCSI disks to replace the single IDE root partition. I figured it's time for a fresh install as the system was originally RedHat 6.2 that got merged by hand with EnGarde 1.0 which got manually upgraded to EnGarde Professional 1.5 and has been lingering in existence like this for the better part of 5 years. So I figured I would give Ubuntu Server (7.04) a shot.

The installation was rough. Quite a challenge getting the Software RAID arrays to play friendly. I had to do a number of software RAID things manually. Anyway, once I got it working and built one of the two new (refurbed) SCSI disks I bought died. So I started everything over from scratch with two new hard drives.

Everything is finally installed and the server is running off of a wireless connection and currently resides in my garage. I have since upgraded to Ubuntu 7.10 without issue. One last "issue" I am sure will come. The garage is nice and cold in the winter; however, during the summer months to come I am expecting to possibly see some hard drive failures. As of right now the machine is way too loud to keep in the house. We don't have a basement or any secluded areas. This is the first time since January 2001 that Demandred has not also served as a router. When I moved it from my parents' place they bought their first wireless router. I had purchased one several months earlier.

I also finally spell checked this file.

AMD Athlon 800MHz (tbird)
1.5GB RAM
2x Seagate Barracuda 18.2GB (md0 -- /home)
2x Seagate Barracuda 18.2GB (md1 -- /)
100BaseT Realtek 8139
Ra-Link RT2561/RT61 802.11g PCI
Adaptec AIC-7881U
NVidia TNT2 16Mb AGP Video
Mitsumi 24x/12x CD/RW
ATX Full-tower
Ubuntu Server 7.10 

July 2008

It's been a year since I've posted anything regarding Demandred and the good news is there's been none. Demandred is sitting in my garage just happily doing it's job. Everything is good so far, knock on wood.

I have a general hardware status page for demandred here.

I've had more headaches from my other computers, failed hard drives, bad memory, two dead laptops, etc. then I'd ever like to tackle, but demandred for once is headache free.

AMD Athlon 800MHz (tbird)
1.5GB RAM
2x Seagate Barracuda 18.2GB (md0 -- /home)
2x Seagate Barracuda 18.2GB (md1 -- /)
100BaseT Realtek 8139
Ra-Link RT2561/RT61 802.11g PCI
Adaptec AIC-7881U
NVidia TNT2 16Mb AGP Video
Mitsumi 24x/12x CD/RW
ATX Full-tower
Ubuntu Server 7.10

September 2008

The server appeared to go down, so I connected to it via the serial connection and confirmed the network was down. The system was refusing to recognize the wireless card. After a reboot and the wireless NIC failed to appear. So I swapped the wireless card with another one and the server failed to boot. Being that I had only the serial connection I needed to hook up a monitor at this point.

After some troubleshooting I discovered that one of the SCSI disks in md1 is failing to read properly. It IDs itself to the controller but the controller can't access it beyond that. The same problem with the linux SCSI driver. The RAID array refused to come up with only one disk.

For now I changed 'root=/dev/md1' to 'root=/dev/sdd2' and all is running on the single disk. I have a cold spare SOMEWHERE. I bought it a LONG time ago, and it made several moves. I THINK it's at my parents' house. I'll have to check it out the next time I am there.

These are the hardware errors I am seeing on the bad disk:

# dmesg | grep -i sdb
[  120.340784] sd 0:0:2:0: [sdb] Spinning up disk........................not responding...
[  221.290137] sd 0:0:2:0: [sdb] READ CAPACITY failed
[  221.290147] sd 0:0:2:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE,SUGGEST_OK
[  221.290159] sd 0:0:2:0: [sdb] Sense Key : Not Ready [current] 
[  221.290168] sd 0:0:2:0: [sdb] Add. Sense: Logical unit not ready, cause not reportable
[  221.294715] sd 0:0:2:0: [sdb] Test WP failed, assume Write Enabled
[  221.296227] sd 0:0:2:0: [sdb] Asking for cache data failed
[  221.296285] sd 0:0:2:0: [sdb] Assuming drive cache: write through
[  221.296493] sd 0:0:2:0: [sdb] Attached SCSI disk
I confirmed that all the connection were solid. The SCSI controller gives errors when trying to pull up simple data in the BIOS on the drive and the system BIOS does not recognize that disk as a 'bootable' disk at all. So I'll just have to replace it, assuming I can find my spare. I left the replacement wireless NIC in there.

AMD Athlon 800MHz (tbird)
1.5GB RAM
2x Seagate Barracuda 18.2GB (md0 -- /home)
2x Seagate Barracuda 18.2GB (md1 -- /) (Running on one disk)
100BaseT Realtek 8139
Ra-Link RT2500 802.11g PCI (rev 01)
Adaptec AIC-7881U
NVidia TNT2 16Mb AGP Video
Mitsumi 24x/12x CD/RW
ATX Full-tower
Ubuntu Server 7.10

November 2008

Increasing wireless problems and complex hardware configurations finally just pushed me to the point of tearing everything apart and starting from scratch. Sort of. I had a spare 1700+ system. Full system with an 80gb hard drive. I upgraded my main system to a new hard drive. It previously had a 160gb and a 200gb drive in it. The 200gb drive was a backup drive for the house.

So I took 20gb of the 80gb for '/' and then grabbed the remaining 60gb and 60gb from the 160gb a created a stripped RAID array for '/home'. The remaining space on the 160gb (100gb) was used for swap and for a network drive I used to run off of my desktop machine. The 200gb drive is remaining as a backup.

The new system is an Athlon 1700+ with 1.5GB of RAM, it's in a smaller case next to my desk and is WIRED to the network on a 1GB connection. Yay. The system was installed with Ubuntu 8.04LTS. Lets see how long this lasts...

So after 4+ years, the refurbished SCSI drives I bought are still running just fine. However, I am not using them simply for reduced complexity in my system. If I continue to have hard drive problems, I will definitely consider putting these back in. The big downside is they draw a lot of power, generate a lot of heat and are extremely noisy.

AMD Athlon 1700+
1.5GB RAM
80GB ( /, /home (md0) )
Seagate 160GB ( /home (md0), network drive and swap )
Maxtor 200GB ( backup drive)
Intel 8254 Gigabit Ethernet (e1000)
ATI Radeon 9200 64Mb
NEC 12x DVD Burner
ATX Mid-Tower
Ubuntu Server 8.04LTS

May 25th 2009 (Memorial Day)

Memorial Day morning I got up, got Connor ready and made a cup of coffee. Moments after sitting down at my computer I hear a series of fizzles and pops. Then the hard drive light on demandred goes solid red. I switch over to demandred and it's dead.

After a couple reboot retries I open demandred up and discover 11 capacitors on the motherboard are blown. Oh well, I am not too surprised and not that upset. The motherboard was 8 years old. I have a spare P4 2ghz box but I only have 512MB I can stuff in it. That's just enough to squeeze by but I have some virtual machines running on Demandred now and what-not. So I looked around and scored a new Athlon 64 3200+ w/ motherboard for $36 + shipping. So I bought that.

In the interim I was going to put the P4 but I couldn't get it running. I didn't have time to mess with demandred much as we were getting everything ready for our annual Memorial Day BBQ.

When I mentioned this to Brent he brought a spare computer with him to the BBQ. Later that evening, after helping us clean up (thanks again Brent!) he tossed the motherboard/processor/memory in to demandred. After some fiddling around he got it all working. The new motherboard and processor should come by this weekend but I will be using my spare time to bottle my beer. So I'll have Brent's hardware running in here for 2-3 weeks. It has only 512MB of RAM for now but I can make do for a bit without my virtual boxen running. Thanks again Brent for all your help and the hardware loan :)

I also finally added some navigation at the top and this passed through the w3c validator, yay! :p

  AMD Athlon 1800+
  512MB RAM
  80GB ( /, /home (md0) )
  Seagate 160GB ( /home (md0), network drive and swap )
  Maxtor 200GB ( backup drive)
  Intel 8254 Gigabit Ethernet (e1000)
  ATI Radeon 8500 LE
  NEC 12x DVD Burner
  ATX Mid-Tower
  Ubuntu Server 8.04LTS

June 6th 2009

A new demandred is born. After receiving the new 3200+ motherboard and chip I realized I needed to upgrade the power supply. I bought a new PC Power & Cooling Silencer PPCS370X 370W power supply. I hope this lasts as long as the last one. This PS came in towards the end of the week. So Saturday morning I tossed it all together.

Everything went mostly smoothly except for the RAM. I have 1x2GB of DDR400 (pc3200) high density RAM. Unfortunately the motherboard is not compatible with this. Brent's DDR333 512MB stick works fine it however. So that is what it is running off of for now.

Everything appears to be running well and is quite fast except when swapping. I will be upgrading the RAM at some point in the not so distant future. For now, I am really happy to have Demanded stable again. Current system stats:

  AMD Athlon 64 3200+
  512MB DDR333 RAM
  Western Digital (WD800JB-00JJ) 80GB ( /, /home (md0) )
  Seagate (ST3160023A) 160GB ( /home (md0), network drive and swap )
  Maxtor (6L200P0) 200GB ( backup drive)
  Intel 8254 Gigabit Ethernet (e1000)
  NEC (ND-3520A) 12x DVD Burner
  nVidia GeForce 6100 (onboard)
  ATX Mid-Tower w/ upgraded power supply
  Ubuntu Server 8.04LTS

June 22nd 2009

I was hoping this entry would be regarding something along the lines of a RAM upgrade. Unfortunately, it's not. Some cracker from Korea or China broke in to demandred and installed a rootkit. According to rkhunter I had SHV4 and SHV5 rootkits installed on my system. I detected a problem with the system late Saturday but dismissed it as something I possibly screwed up. I didn't look at it again until Monday morning and promptly took Demandred off the main network and started to investigate. I also checked the other 4 systems running Linux to see if there were any problems, there were none. After further investigation, I discovered it was Linux.RST.B-1 that installed on my server.

After investigating I saw a number of attacks from Korean and Chinese owned IP addresses. So I am concluding the attack came from there. I am uncertain how they got in exactly but I have to admit I was getting lax with security and leaving a lot of ports opened. So I have things locked down much better now.

So tonight I re-installed Ubuntu but upgraded to version 9.04 and to the 64bit version as well. The install went smoothly and I managed to bring a number of configuration files over from my backups and got 95% of everything working in less than an hour. So I ended up with a minor OS upgrade through all this. Hopefully my next entry will concern a memory upgrade :)

  AMD Athlon 64 3200+
  512MB DDR333 RAM
  Western Digital (WD800JB-00JJ) 80GB ( /, /home (md0) )
  Seagate (ST3160023A) 160GB ( /home (md0), network drive and swap )
  Maxtor (6L200P0) 200GB ( backup drive)
  Intel 8254 Gigabit Ethernet (e1000)
  NEC (ND-3520A) 12x DVD Burner
  nVidia GeForce 6100 (onboard)
  ATX Mid-Tower w/ upgraded power supply
  Ubuntu Server 9.04 (64bit)

November 17th 2009

A friend of mine managed to secure a really nice, APC 1200VA 780 Watt UPS for me to have, thanks Sean! I hooked this up to demandred as well as my desktop, router and DSL modem. I haven't hooked up demandred to interact with the UPS as of yet but I think I'll investigate using 'nut' to manage everything.

I also wanted to make a note. The power supply I bought a few months back works incredibly well. It's extremely quiet and generates very little heat. The old demandred set up was very hot over the rear power supply, especially with my switch, router and DSL modem sitting atop it. This new power supply barely sits above room temperature.

  AMD Athlon 64 3200+
  512MB DDR333 RAM
  Western Digital (WD800JB-00JJ) 80GB ( /, /home (md0) )
  Seagate (ST3160023A) 160GB ( /home (md0), network drive and swap )
  Maxtor (6L200P0) 200GB ( backup drive)
  Intel 8254 Gigabit Ethernet (e1000)
  NEC (ND-3520A) 12x DVD Burner
  nVidia GeForce 6100 (onboard)
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 9.04 (64bit)

June 8th 2010

I hope I am not jinxing anything by posting this. Demandred broke 200 days consecutive uptime.

 10:36:25 up 202 days, 11:16, 29 users,  load average: 0.14, 0.10, 0.02

Since I have the UPS in and some stable hardware things have been humming along very smoothly. We've lost power a number of times at the house and the UPS kicked right in no problem and with mostly all new hardware in demandred things have been running smooth. That new power supply generates very little heat as well. The only issue I had lately was my Linksys router finally gave out after 5 years of use.

  AMD Athlon 64 3200+
  512MB DDR333 RAM
  Western Digital (WD800JB-00JJ) 80GB ( /, /home (md0) )
  Seagate (ST3160023A) 160GB ( /home (md0), network drive and swap )
  Maxtor (6L200P0) 200GB ( backup drive)
  Intel 8254 Gigabit Ethernet (e1000)
  NEC (ND-3520A) 12x DVD Burner
  nVidia GeForce 6100 (onboard)
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 9.04 (64bit)

August 3rd 2010

So after 258 days uptime, demandred was accidently rebooted today. So I decided to take this time to upgrade from Ubuntu 9.04 to 9.10 and upgrade the memory. I upgraded the OS because 9.04 is end of life in a couple of months. I want to go to the 10.04 LTS but have to go to 9.10 first to get there. I also am waiting until August 12th for 10.04.1 point release to come out before upgrading to 10.04. So they'll be one more reboot.

As far as memory goes, I came across a 1GB 400mhz stick of RAM so I dropped that in for a total of 1.5GB of RAM. I also found a 512MB stick of 400MHz RAM to replace the 333MHz stick in there now. No more swapping and more speed...

  AMD Athlon 64 3200+
  1.5GB DDR 400MHz RAM
  Western Digital (WD800JB-00JJ) 80GB ( /, /home (md0) )
  Seagate (ST3160023A) 160GB ( /home (md0), network drive and swap )
  Maxtor (6L200P0) 200GB ( backup drive)
  Intel 8254 Gigabit Ethernet (e1000)
  NEC (ND-3520A) 12x DVD Burner
  nVidia GeForce 6100 (onboard)
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 9.10 (64bit)

September 5th 2010

I've been having problems with demandred randomly tossing out kernel panics. I think it's from the memory. I've had quite a few reboots. So I put the old memory back in. I also took this opportunity to perform the 10.04.1 LTS upgrade which went without a hitch. I'll have to see what I can do about the memory.

  AMD Athlon 64 3200+
  512MB DDR 333MHz RAM
  Western Digital (WD800JB-00JJ) 80GB ( /, /home (md0) )
  Seagate (ST3160023A) 160GB ( /home (md0), network drive and swap )
  Maxtor (6L200P0) 200GB ( backup drive)
  Intel 8254 Gigabit Ethernet (e1000)
  NEC (ND-3520A) 12x DVD Burner
  nVidia GeForce 6100 (onboard)
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 10.04.1 LTS (64bit)



November 28th 2010

Finally got some decent memory and tossed it in. Bumped demandred up to 1GB now. This will alleviate the heavy swapping. It was getting so bad it would take a good 15 seconds to get a prompt after log in.

On a side note, today is my son's 3rd birthday. Happy Birthday Connor!

  AMD Athlon 64 3200+
  1GB 400Mhz DRAM
  Western Digital (WD800JB-00JJ) 80GB ( /, /home (md0) )
  Seagate (ST3160023A) 160GB ( /home (md0), network drive and swap )
  Maxtor (6L200P0) 200GB ( backup drive)
  Intel 8254 Gigabit Ethernet (e1000)
  NEC (ND-3520A) 12x DVD Burner
  nVidia GeForce 6100 (onboard)
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 10.04.1 LTS (64bit)

December 10th 2011

Why has memory been such an issue??? Even with 1GB in demandred it was swapping like crazy. As it turns out, Brendan's old desktop died a few months back and he (through AJ) donated me his 2GB of memory. I finally got around to getting it in there. Two nice, 1GB Corsair sticks. Thanks again Brendan!

  AMD Athlon 64 3200+
  2GB 400Mhz DRAM (2x1GiB)
  Western Digital (WD800JB-00JJ) 80GB ( /, /home (md0) )
  Seagate (ST3160023A) 160GB ( /home (md0), network drive and swap )
  Maxtor (6L200P0) 200GB ( backup drive)
  Intel 8254 Gigabit Ethernet (e1000)
  NEC (ND-3520A) 12x DVD Burner
  nVidia GeForce 6100 (onboard)
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 10.04.1 LTS (64bit)

March 30th, 2013

I finally broke down and got some real hard drive space. I removed all 3 of the old drives and migrated over to two 2TB Seagate Barracudas. One drive is for backups and the other runs the system. Hopefully these will last quite a while. I also tossed in a Rosewell USB card reader in place of my 3.5" floppy drive.

  AMD Athlon 64 3200+
  2GB 400Mhz DRAM (2x1GiB)
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( /, /home )
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( /home/backup )
  Intel 8254 Gigabit Ethernet (e1000)
  NEC (ND-3520A) 12x DVD Burner
  nVidia GeForce 6100 (onboard)
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 10.04.4 LTS (64bit)



March 30th, 2014

Pretty much, to do the day when I upgraded demandred to brand new hard drives, one failed. Hard drives really are the most god awful things in the world. Second one this year, one of my desktop hard drives failed back in January. These were the errors I was receiving for the failed demandred drive:

[583225.359901] ata3.00: status: { DRDY ERR }
[583225.364428] ata3.00: error: { UNC }
[583225.368918] ata3.00: failed command: READ FPDMA QUEUED
[583225.373422] ata3.00: cmd 60/50:78:17:83:e2/00:00:00:00:00/40 tag 15 ncq 40960 in
[583225.373425]          res 41/40:50:47:07:80/40:00:00:00:00/40 Emask 0x9 (media error)
[583225.382484] ata3.00: status: { DRDY ERR }
[583225.387005] ata3.00: error: { UNC }
[583225.391523] ata3: hard resetting link
[583225.391527] ata3: nv: skipping hardreset on occupied port
[583225.880051] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[583225.940433] ata3.00: configured for UDMA/133
[583225.940481] sd 2:0:0:0: [sda] Unhandled sense code
[583225.940486] sd 2:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[583225.940492] sd 2:0:0:0: [sda] Sense Key : Medium Error [current] [descriptor]
[583225.940500] Descriptor sense data with sense descriptors (in hex):
[583225.940504]         72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00 
[583225.940519]         00 80 07 47 
[583225.940526] sd 2:0:0:0: [sda] Add. Sense: Unrecovered read error - auto reallocate failed
[583225.940535] sd 2:0:0:0: [sda] CDB: Read(10): 28 00 00 80 01 bf 00 00 50 00
[583225.940549] end_request: I/O error, dev sda, sector 8389055
[583225.945154] ata3: EH complete

I made the mistake of not RAIDing my drives together so this turned in to quite a headache. What I ended up doing was yanking the bad drive, which housed '/' and '/home' and using the backup drive in place of it. I had a spare 3TB drive I made the new backup drive. I also changed things over to RAID1; however, with a missing drive. So the array is running in degraded mode. However, this will allow me to add the replacement 2TB drive in to the system with minimal hassle. Though, this motherboard only had 2 SATA slots, so I will need to order an additional SATA controller. As it stands now, this is my RAID array status:

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md127 : active raid1 sda1[0]
      30121728 blocks [2/1] [U_]
      
md126 : active raid1 sda2[0]
      1922048640 blocks [2/1] [U_]
      
unused devices: <none>

I had many difficulties moving root to a bootable RAID array, so after much frustration I decided to just install Ubuntu 12.04.4 LTS. Especially considering the support for 10.04 LTS ends this time next year and I would have needed to upgrade at that time regardless. I'm now covered until April of 2017, but might move to Trusty before than.

  AMD Athlon 64 3200+
  2GB 400Mhz DRAM (2x1GiB)
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( / (md127), /home (md126) )
  Seagate (ST3000DM001-1CH1) Barracuda 3TB ( /home/backup )
  Intel 8254 Gigabit Ethernet (e1000)
  NEC (ND-3520A) 12x DVD Burner
  nVidia GeForce 6100 (onboard)
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 12.04.4 LTS (64bit)

April 13th, 2014

I received the replacement drive from Seagate and installed it. However, given I have 3 SATA devices in the system now there are only 2 SATA ports on the motherboard. So I had to purchase a PCIe SATA Card. I found the IO Crest 2 Port SATA III PCIe 1x card on Amazon Prime for a good price. The chipset is the Asmedia 1061 (ASM1061) which I read has very good support in Linux. So after installing this card and the new drive the system found both with no extra work on my part. I simply used sfdisk to copy the partition layout of the first drive and then via mdadm added the two partitions to the existing array. The current RAID status is as follows:

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md126 : active raid1 sda2[1] sdb2[0]
      1922048640 blocks [2/2] [UU]
      
md127 : active raid1 sda1[1] sdb1[0]
      30121728 blocks [2/2] [UU]
      
unused devices: <none>

So now everything looks really good in the system. I do have one issue, the case I am using is a case I've had since circa 2000 which became demandred's case back in November 2008 so it lacks available drive bays without stacking the drives directly on top of eachother, which I want to avoid because of heat. So I currently have one drive just resting at the bottom of the case. Either it's time to upgrade to a newer case or come up with a more solid solution. The updated system specs are below.

  AMD Athlon 64 3200+
  2GB 400Mhz DRAM (2x1GiB)
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( / (md127), /home (md126) )
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( / (md127), /home (md126) )
  Seagate (ST3000DM001-1CH1) Barracuda 3TB ( /home/backup )
  Intel 8254 Gigabit Ethernet (e1000)
  ASMedia Technology Inc. ASM1062 Serial ATA Controller
  NEC (ND-3520A) 12x DVD Burner
  nVidia GeForce 6100 (onboard)
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 12.04.4 LTS (64bit)

December 19th, 2014

So the other 2GB drive in the array failed. Really can't stand this crap. I just sent the drive back for an RMA about a week ago and put the new one in the other night. Today I got it back in the raid array. Just for my own reference, this is the commands I used to bring the new drive in to the array:

# sfdisk -d /dev/sdf | sfdisk /dev/sda
# mdadm --manage /dev/md126 --add /dev/sda2
# mdadm --manage /dev/md127 --add /dev/sda1

Really nothing different in the hardware specs as Seagate sent me an exact replacement drive:

  AMD Athlon 64 3200+
  2GB 400Mhz DRAM (2x1GiB)
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( / (md127), /home (md126) )
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( / (md127), /home (md126) )
  Seagate (ST3000DM001-1CH1) Barracuda 3TB ( /home/backup )
  Intel 8254 Gigabit Ethernet (e1000)
  ASMedia Technology Inc. ASM1062 Serial ATA Controller
  NEC (ND-3520A) 12x DVD Burner
  nVidia GeForce 6100 (onboard)
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 12.04.4 LTS (64bit)

April 2015

While I am not certain of the date as I was starting a new job and had a lot going on, I lost two more hard drives. One of the 2TB drives in my RAID array was reporting failed and the 3TB backup drive went into read only mode. After running through the drives I determined that the 3TB drive was completely toasted but the 2TB drive only had some bad sectors. However, I was preparing a complete overhaul of Demandred so I didn't attend to this immedietely and left the RAID array in degraded mode. I hooked up an external 2TB drive to handle my backups.

  AMD Athlon 64 3200+
  2GB 400Mhz DRAM (2x1GiB)
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( / (md127), /home (md126) )
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( / (md127), /home (md126) ) *DEAD*
  Seagate (ST3000DM001-1CH1) Barracuda 3TB ( /home/backup ) *DEAD*
  Seagate (Verbatim chassis) 2TB USB drive
  Intel 8254 Gigabit Ethernet (e1000)
  ASMedia Technology Inc. ASM1062 Serial ATA Controller
  NEC (ND-3520A) 12x DVD Burner
  nVidia GeForce 6100 (onboard)
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 12.04.4 LTS (64bit)

May 23rd, 2015

This is the largest upgrade Demandred has ever received. For the first time ever I purchased a good deal of new hardware including a new case, DVD burner, and front panel controller. The system got an upgrade from the AMD single code 3200+ to a dual core 5000+ and a memory bump from 2GB to 6GB DDR2. To manage the failing drives I'm increasing airflow with extra 120mm fans controlled by a fan controller that installs in a 5.25" bay, which also comes with a USB 3 PCI-E controller as well.

The hard drives are finally settled as well. The failed 2TB drive had only bad sectors. I did a badblocks test on it and when I reformatted it I had it skip the bad sectors. This is used as the second disk in my RAID 1 array. A bought a new 2TB drive and made this my backup drive.

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md126 : active raid1 sdb2[0] sda2[1]
      1922048640 blocks [2/2] [UU]
      
md127 : active raid1 sdb1[0] sda1[1]
      30121728 blocks [2/2] [UU]

Also, for the first time, I have X running on demandred. Since my new job is working from home I use demandred to run all my irc windows, pidgin, skype, music (pithos and audacious) and a few other things.

And a few pictures of the new setup:

  AMD Athlon X2 64 5000+
  6GB DDR2 (2x2GiB + 2x1GiB)
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( / (md127), /home (md126) )
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( / (md127), /home (md126) )
  Seagate (ST32000644NS) Constellation ES 2TB ( /home/backup )
  nVidia GeForce GT 630
  LineOn iHAS124 24x DVD Burner
  5.25" USB 3.0 MultiMedia Front Panel PCI-E card reader and fan controller
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 12.04.5 LTS (64bit)



June 23rd, 2016

375 days uptime. Coincidentally, also the number of days one of my hard drives took to reach failure.

[Thu Jun 23 14:19:25 2016] ata1.00: status: { DRDY ERR }
[Thu Jun 23 14:19:25 2016] ata1.00: error: { ABRT }
[Thu Jun 23 14:19:25 2016] ata1.00: configured for UDMA/133
[Thu Jun 23 14:19:25 2016] ata1: EH complete
[Thu Jun 23 14:19:25 2016] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
[Thu Jun 23 14:19:25 2016] ata1.00: irq_stat 0x40000001
[Thu Jun 23 14:19:25 2016] ata1.00: failed command: READ DMA
[Thu Jun 23 14:19:25 2016] ata1.00: cmd c8/00:02:00:00:00/00:00:00:00:00/e0 tag 12 dma 1024 in

  AMD Athlon X2 64 5000+
  6GB DDR2 (2x2GiB + 2x1GiB)
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( / (md127), /home (md126) )
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( / (md127), /home (md126) )
  Seagate (ST32000644NS) Constellation ES 2TB ( /home/backup )
  nVidia GeForce GT 630
  LineOn iHAS124 24x DVD Burner
  5.25" USB 3.0 MultiMedia Front Panel PCI-E card reader and fan controller
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 12.04.5 LTS (64bit)



January 10th, 2017

After replacing that last hard drive in June, I had the next one fail. At least I am becoming a pro at swapping drives and moving them around in their RAID arrays. Anyway, I'm starting to get done with these drive failures and am thinking of moving as much as possible off the server. However, I still need the server. So in the interm, I moved the root partition and the home directories, excluding the 2TB network share onto a pair of RAID 1 SSDs. The system is so much more responsive now. I left the old network share on the remaining good disk. I have this backed up in two additional places. So hopefully this will last until I save up some cash for my final push to move it off.

  AMD Athlon X2 64 5000+
  6GB DDR2 (2x2GiB + 2x1GiB)
  PNY CS1311 240GB x2 ( / (md126), /home (md127) )
  Seagate (ST2000DM001-1CH1) Barracuda 2TB ( /home/share (md0) )
  Seagate (ST32000644NS) Constellation ES 2TB ( /home/backup )
  nVidia GeForce GT 630
  LineOn iHAS124 24x DVD Burner
  5.25" USB 3.0 MultiMedia Front Panel PCI-E card reader and fan controller
  ATX Mid-Tower
  PC Power & Cooling Silencer PPCS370X 370w Power Supply
  APC Back-UPS 1200VA
  Ubuntu Server 12.04.5 LTS (64bit)



Last Updated -- January 10th, 2017

Valid XHTML 1.0 Transitional