Hi Austin,
>It is perplexing ... I chocked it up to the following statement in the
>fdisk manpage:
Indeed this is weird. I've tested the 32bit case, I'll try the 64bit case later on. Dan mentioned this earlier and it could be a problem with 64bit fdisk for whatever reason. In any event, knowing that sfdisk fixes the problem is good enough for me and we'll certainly be using this to implement the expandvolume functionality (because we *have* to).
>As for the expandvolume command, I have only taken a look at the
>VolumeCreator class and thought about it a bit. If I find the urge to
>implement it I will let you know before I begin. Looking at it, I was
>thinking of implementing an "expand" method along the lines its existing
>"create" method.
OK great, just let me know. I agree, adding an expand method to the VolumeCreator class is probably the way to go. I'll post to the list whenever I have something to test.
~Justin
________________________________________
From: Austin Godber [godber_at_uberhip.com]
Sent: Friday, August 13, 2010 4:20 PM
To: Justin Riley
Cc: starcluster_at_mit.edu
Subject: Re: [Starcluster] enlarge volume
Hi Justin, I am/was puzzled as well. But last night when I tried this
myself, step #5 caused step #6 to result in a missing superblock error
as Dan reported.
When removing and creating the partition, I deleted it, then created a
new one accepting the default start and stop values. I did not
explicitly set a partition type, though I assume (this could be false)
that the partition type would be set to Linux. I coulda messed
something up, but I tried at least twice with the same results.
The nice thing about the sfdisk command is that, as you point out, it is
non-interactive, so we all know that we are doing the exact same thing.
It is perplexing ... I chocked it up to the following statement in the
fdisk manpage:
"There are several *fdisk programs around. Each has its
problems and strengths. Try them in the order cfdisk, fdisk, sfdisk.
(Indeed, cfdisk is a beautiful program that has strict requirements on
the partition tables it accepts, and produces high quality partition
tables. Use it if you can. fdisk is a buggy program that does fuzzy
things - usually it happens to produce reasonable results. Its single
advantage is that it has some support for BSD disk labels and other
non-DOS partition tables. Avoid it if you can. sfdisk is for hackers
only - the user interface is terrible, but it is more correct than fdisk
and more powerful than both fdisk and cfdisk. Moreover, it can be used
noninteractively.)"
As for the expandvolume command, I have only taken a look at the
VolumeCreator class and thought about it a bit. If I find the urge to
implement it I will let you know before I begin. Looking at it, I was
thinking of implementing an "expand" method along the lines its existing
"create" method.
Austin
On 08/13/2010 03:39 PM, Justin Riley wrote:
> Hi Guys,
>
> Not sure what the issue is for you, but I just resized one of my testvolumes in 5 min like so:
>
> 1. Snapshot the volume you wish to resize
> 2. Create a new, larger volume based off the snapshot in step 1
> 3. Attach the new volume to an instance in the same zone as the new volume (on /dev/sdz for this example)
> 4. Login as root
> 5. fdisk /dev/sdz, remove the partition, recreate the partition using all cylinders
> 6. e2fsck -f /dev/sdz1
> 7. resize2fs -p /dev/sdz1
> 8. fdisk -l and confirm that blocks/1024 == the new volume size
> 8. mkdir /mnt/ebs
> 9. mount /dev/sdz1 /mnt/ebs
> 10. cd /mnt/ebs/
> 11. ls and see that your data is there on the new larger volume
>
> I did this using the standard 32bit StarCluster Ubuntu 9.10 Karmic AMI.
>
> Austin, I noticed that you're using the same echo/sfdisk command that I used inside of createvolume. I'm not sure how, if at all, this would be different from running fdisk manually given that fdisk worked for me no problems but if it works for you, great. I agree that it would be good to create an expandvolume command that creates a new larger volume automatically. The sfdisk route is the only way to do the partitioning non-interactively so this would certainly be the approach needed in expandvolume similar to createvolume. If you already have something, I'd be interested in merging the code on github assuming you do this in your fork, otherwise I will create it when I get time.
>
> ~Justin
> ________________________________________
> From: starcluster-bounces_at_MIT.EDU [starcluster-bounces_at_MIT.EDU] On Behalf Of Austin Godber [godber_at_uberhip.com]
> Sent: Thursday, August 12, 2010 8:31 PM
> To: starcluster_at_mit.edu
> Subject: Re: [Starcluster] enlarge volume
>
> Alright, I have it running, here's the short version, commands only, no output, run as root, disk is attached at /dev/sdf with one partition:
>
>
> Check the filesystem (might want to use -f to force a real check)
> # fsck /dev/sdf1
>
> Repartition with sfdisk, start at the first cylinder end at the last
> # echo ",,L" | sfdisk /dev/sdf
>
> Resize the ext3 filesystem
> # resize2fs /dev/sdf1
>
> Recheck the filesystem
> # fsck -f /dev/sdf1
>
> Now mount and check that the data is there and the size is correct with df -h.
>
> Let me know if it works for you.
>
> - Austin
>
> Here is my verbose version, with output.
>
> # fsck /dev/sdf1
> fsck from util-linux-ng 2.16
> e2fsck 1.41.9 (22-Aug-2009)
> /dev/sdf1: clean, 11/196608 files, 29901/785176 blocks
>
> # sfdisk -l /dev/sdf
>
> Disk /dev/sdf: 652 cylinders, 255 heads, 63 sectors/track
> Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
>
> Device Boot Start End #cyls #blocks Id System
> /dev/sdf1 0+ 390 391- 3140707 83 Linux
> /dev/sdf2 0 - 0 0 0 Empty
> /dev/sdf3 0 - 0 0 0 Empty
> /dev/sdf4 0 - 0 0 0 Empty
>
>
> # echo ",,L" | sfdisk /dev/sdf
> Checking that no-one is using this disk right now ...
> OK
>
> Disk /dev/sdf: 652 cylinders, 255 heads, 63 sectors/track
> Old situation:
> Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
>
> Device Boot Start End #cyls #blocks Id System
> /dev/sdf1 0+ 390 391- 3140707 83 Linux
> /dev/sdf2 0 - 0 0 0 Empty
> /dev/sdf3 0 - 0 0 0 Empty
> /dev/sdf4 0 - 0 0 0 Empty
> New situation:
> Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
>
> Device Boot Start End #cyls #blocks Id System
> /dev/sdf1 0+ 651 652- 5237189+ 83 Linux
> /dev/sdf2 0 - 0 0 0 Empty
> /dev/sdf3 0 - 0 0 0 Empty
> /dev/sdf4 0 - 0 0 0 Empty
> Warning: no primary partition is marked bootable (active)
> This does not matter for LILO, but the DOS MBR will not boot this disk.
> Successfully wrote the new partition table
>
> Re-reading the partition table ...
>
> If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
> to zero the first 512 bytes: dd if=/dev/zero of=/dev/foo7 bs=512 count=1
> (See fdisk(8).)
>
>
> # resize2fs /dev/sdf1
> resize2fs 1.41.9 (22-Aug-2009)
> Resizing the filesystem on /dev/sdf1 to 1309297 (4k) blocks.
> The filesystem on /dev/sdf1 is now 1309297 blocks long.
>
> # fsck -f /dev/sdf1
> fsck from util-linux-ng 2.16
> e2fsck 1.41.9 (22-Aug-2009)
> Pass 1: Checking inodes, blocks, and sizes
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> /dev/sdf1: 11/327680 files (0.0% non-contiguous), 38511/1309297 blocks
>
>
>
>
>
>
> On 08/12/2010 08:01 PM, Dan Yamins wrote:
> Let me add one more thing to this thread:
>
> Even on ubuntu, even with my original volume, you can't do "e2fsck" of the "device" , e.g.
>
> $ e2fsck /dev/sdz
> Couldn't find ext2 superblock, trying backup blocks...
> e2fsck: Bad magic number in super-block while trying to open /dev/sdo
> ...
>
> Is that to be expected? In a number of tutorials on resizing EBS volumes, i have seen this done with the devices itself. Is the reason this is failing because its a partitioned drive?
>
>
>
> On Thu, Aug 12, 2010 at 6:43 PM, Dan Yamins<dyamins_at_gmail.com<mailto:dyamins_at_gmail.com>> wrote:
>
> Do you have a snapshot? I have used parted and e2resizefs to expand an ext3 filesystem without trouble.
>
>
>
> So I'm trying parted on a new, clean volume created from the snapshot, attached to /dev/sdm As I explained before, I can't do
>
> $ parted /dev/sdm
> [root_at_domU-12-31-39-0E-B2-61 ~]# parted /dev/sdm
> Error: Error initialising SCSI device /dev/sdm - Invalid argument
>
> AUstin, when you did this did you get that same error?
>
>
> I CAN do parted on the path:
>
> $ parted /dev/sdm1
> GNU Parted 1.8.1
> Using /dev/sdm1
> Welcome to GNU Parted! Type 'help' to view a list of commands.
>
> But then i get this error:
>
> (parted) resize 1 0 500GB
> Error: The location 500GB is outside of the device /dev/sdo1.
>
> Which is course is true ...
>
> (parted) print
> Disk /dev/sdm1: 21.5GB
> Sector size (logical/physical): 512B/512B
> Partition Table: loop
>
> Number Start End Size File system Flags
> 1 0.00kB 21.5GB 21.5GB ext3
>
>
> So I'm not sure where to go from here...
>
> Dan
>
>
>
>
>
>
> _______________________________________________
> Starcluster mailing list
> Starcluster_at_mit.edu<mailto:Starcluster_at_mit.edu>
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
>
>
Received on Fri Aug 13 2010 - 16:43:43 EDT