Move from cooked to raw
Posted in 2000
A DBA on HP-UX 10.20/LVM asked how to move remaining cooked chunks, including rootdbs and the catalog-holding first chunk, to raw devices. Suggestions: take a level-0 archive, shut down, repoint the chunk paths/links to raw devices and restore; or dd the files to raw with the engine down; or (for no downtime) mirror each chunk to the new raw storage, then drop the original primary. Warning raised that some LVMs (AIX, Veritas/Solaris) write a few KB of metadata at the start of a raw device, so an offset may be needed; the posters agreed offset 0 is fine under HP-UX LVM. No single outcome reported, but the dd-while-down and mirror approaches were both endorsed, the choice depending on whether downtime is acceptable.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Topics: Storage & Space Management
I currently have a mixed environment of cooked and raw chunks. I have been slowly moving tables over to raw devices, but the first chunk of the dbspace where the database was created contains system catalog tables, can I migrate those to a raw chunk, what about rootdbs, is it possible to migrate that to a raw space as well? Thanks for any advice. Sent via Deja.com http://www.deja.com/ Before you buy.
hi, i think u couuld do this: take a level0-archieve of ur instance .shut it down.remove ur cooked-files, generate links (named as ur originally cooked-devices but linking now on the raw-devices) and restore ur level0-archive. chris cyclonejls@my-deja.com wrote: > I currently have a mixed environment of cooked and raw chunks. I have > been slowly moving tables over to raw devices, but the first chunk of > the dbspace where the database was created contains system catalog > tables, can I migrate those to a raw chunk, what about rootdbs, is it > possible to migrate that to a raw space as well? > > Thanks for any advice. > > Sent via Deja.com http://www.deja.com/ > Before you buy.
One very important warning. On some systems the first few K of a raw device will be overlayed by system information. You can not simply copy the files from a cooked chunk onto a raw device if your cooked chunk did not have an offset and your raw space requires one. cyclonejls@my-deja.com wrote: > I currently have a mixed environment of cooked and raw chunks. I have > been slowly moving tables over to raw devices, but the first chunk of > the dbspace where the database was created contains system catalog > tables, can I migrate those to a raw chunk, what about rootdbs, is it > possible to migrate that to a raw space as well? > > Thanks for any advice. > > Sent via Deja.com http://www.deja.com/ > Before you buy.
OK, this is my situation. My cooked devices have a 0 offset, what should my offset be for the raw device? Does the size of the offset matter, I would think so? I have HPUX 10.2, using LVM. In article <8omf6k$cta$1@nnrp1.deja.com>, cyclonejls@my-deja.com wrote: > I currently have a mixed environment of cooked and raw chunks. I have > been slowly moving tables over to raw devices, but the first chunk of > the dbspace where the database was created contains system catalog > tables, can I migrate those to a raw chunk, what about rootdbs, is it > possible to migrate that to a raw space as well? > > Thanks for any advice. > > Sent via Deja.com http://www.deja.com/ > Before you buy. > Sent via Deja.com http://www.deja.com/ Before you buy.
I don't think that this is a problem with HP's LVM. cyclonejls@my-deja.com wrote: > OK, this is my situation. My cooked devices have a 0 offset, what > should my offset be for the raw device? Does the size of the offset > matter, I would think so? I have HPUX 10.2, using LVM. > > In article <8omf6k$cta$1@nnrp1.deja.com>, > cyclonejls@my-deja.com wrote: > > I currently have a mixed environment of cooked and raw chunks. I have > > been slowly moving tables over to raw devices, but the first chunk of > > the dbspace where the database was created contains system catalog > > tables, can I migrate those to a raw chunk, what about rootdbs, is it > > possible to migrate that to a raw space as well? > > > > Thanks for any advice. > > > > Sent via Deja.com http://www.deja.com/ > > Before you buy. > > > > Sent via Deja.com http://www.deja.com/ > Before you buy.
Offset 0 is fine for this environment, we've been using it for years. cyclonejls@my-deja.com wrote in message <8onvq6$3rj$1@nnrp1.deja.com>... > > >OK, this is my situation. My cooked devices have a 0 offset, what >should my offset be for the raw device? Does the size of the offset >matter, I would think so? I have HPUX 10.2, using LVM. > > >In article <8omf6k$cta$1@nnrp1.deja.com>, > cyclonejls@my-deja.com wrote: >> I currently have a mixed environment of cooked and raw chunks. I have >> been slowly moving tables over to raw devices, but the first chunk of >> the dbspace where the database was created contains system catalog >> tables, can I migrate those to a raw chunk, what about rootdbs, is it >> possible to migrate that to a raw space as well? >> >> Thanks for any advice. >> >> Sent via Deja.com http://www.deja.com/ >> Before you buy. >> > > >Sent via Deja.com http://www.deja.com/ >Before you buy.
How exactly do you 'move' the data from cooked to raw? Is it dd? Please be very careful moving those chunks over. The best migration is the archive/restore approach mentioned earlier. But it sounds like you have already started. Perhaps you could do incremental warm restores for the rest of the system from this point further. Or maybe just a full archive/restore. If you have been using dd, make sure the instance was DOWN during this copy. Otherwise, you most likely have/will have a corrupted system. Another approach I have seen customer do, is mirror all chunks to the new storage. Then mark offline/remove the primary chunks. The system is always on the secondary mirroed chunks. Its a neat way to migrate with no downtime. Ray cyclonejls@my-deja.com wrote: > > OK, this is my situation. My cooked devices have a 0 offset, what > should my offset be for the raw device? Does the size of the offset > matter, I would think so? I have HPUX 10.2, using LVM. > > In article <8omf6k$cta$1@nnrp1.deja.com>, > cyclonejls@my-deja.com wrote: > > I currently have a mixed environment of cooked and raw chunks. I have > > been slowly moving tables over to raw devices, but the first chunk of > > the dbspace where the database was created contains system catalog > > tables, can I migrate those to a raw chunk, what about rootdbs, is it > > possible to migrate that to a raw space as well? > > > > Thanks for any advice. > > > > Sent via Deja.com http://www.deja.com/ > > Before you buy. > > > > Sent via Deja.com http://www.deja.com/ > Before you buy.
In article <39AFC7BA.2865BACE@informix.com>,
Ray Canuel <rayc@informix.com> wrote:
> How exactly do you 'move' the data from cooked to raw? Is it dd?
>
> Please be very careful moving those chunks over. The best migration
> is the archive/restore approach mentioned earlier. But it sounds like
> you have already started. Perhaps you could do incremental warm
> restores
> for the rest of the system from this point further. Or maybe just a
> full
> archive/restore.
>
> If you have been using dd, make sure the instance was DOWN during this
> copy. Otherwise, you most likely have/will have a corrupted system.
>
> Another approach I have seen customer do, is mirror all chunks to the
> new
> storage. Then mark offline/remove the primary chunks. The system is
> always
> on the secondary mirroed chunks. Its a neat way to migrate with no
> downtime.
>
> Ray
>
[snip]
Ray,
I am using dd as my migration method. I have seen several posts that
say that this method works great. The only caveat that has been
indicated to me is when using a raw device, the first few bytes could
be used for system information. I am using HPUX 10.2, LVM, so this
shouldnt be an issue. I take my instance down before doing dd. Should
I run oncheck to ensure integrity?
I do like your suggestion about mirroring all chunks, then
downing/removing the primary chunk... Does the secondary chunk become
the primary chunk, could I mirror the chunk again if needed? Any other
issues with this method that should be considered? When mirroring, how
will I know that the chunk has completed mirroring?
Thanks for the advice.
Sent via Deja.com http://www.deja.com/
Before you buy.
A bit about the offset. It's more like 16K or so that needs to be offset,
if the offset is needed... ;-)
cyclonejls@my-deja.com wrote:
> In article <39AFC7BA.2865BACE@informix.com>,
> Ray Canuel <rayc@informix.com> wrote:
> > How exactly do you 'move' the data from cooked to raw? Is it dd?
> >
> > Please be very careful moving those chunks over. The best migration
> > is the archive/restore approach mentioned earlier. But it sounds like
> > you have already started. Perhaps you could do incremental warm
> > restores
> > for the rest of the system from this point further. Or maybe just a
> > full
> > archive/restore.
> >
> > If you have been using dd, make sure the instance was DOWN during this
> > copy. Otherwise, you most likely have/will have a corrupted system.
> >
> > Another approach I have seen customer do, is mirror all chunks to the
> > new
> > storage. Then mark offline/remove the primary chunks. The system is
> > always
> > on the secondary mirroed chunks. Its a neat way to migrate with no
> > downtime.
> >
> > Ray
> >
> [snip]
>
> Ray,
>
> I am using dd as my migration method. I have seen several posts that
> say that this method works great. The only caveat that has been
> indicated to me is when using a raw device, the first few bytes could
> be used for system information. I am using HPUX 10.2, LVM, so this
> shouldnt be an issue. I take my instance down before doing dd. Should
> I run oncheck to ensure integrity?
>
> I do like your suggestion about mirroring all chunks, then
> downing/removing the primary chunk... Does the secondary chunk become
> the primary chunk, could I mirror the chunk again if needed? Any other
> issues with this method that should be considered? When mirroring, how
> will I know that the chunk has completed mirroring?
>
> Thanks for the advice.
>
> Sent via Deja.com http://www.deja.com/
> Before you buy.
If these are indeed COOKED devices and not filesystem files then all you have to do is: 0) take a level 0 archive because better safe than sorry 1) shutdown 2) relink the chunkpaths that now point to the COOKED devices to now point to the RAW device file with the same major/minor device numbers. 3) restart the engine. Remember that the RAW and COOKED device files point to the IDENTICAL disk space they just block it differently and cache or do not cache. Been there, done this, it works Art S. Kagel cyclonejls@my-deja.com wrote: > > OK, this is my situation. My cooked devices have a 0 offset, what > should my offset be for the raw device? Does the size of the offset > matter, I would think so? I have HPUX 10.2, using LVM. > > In article <8omf6k$cta$1@nnrp1.deja.com>, > cyclonejls@my-deja.com wrote: > > I currently have a mixed environment of cooked and raw chunks. I have > > been slowly moving tables over to raw devices, but the first chunk of > > the dbspace where the database was created contains system catalog > > tables, can I migrate those to a raw chunk, what about rootdbs, is it > > possible to migrate that to a raw space as well? > > > > Thanks for any advice. > > > > Sent via Deja.com http://www.deja.com/ > > Before you buy. > > > > Sent via Deja.com http://www.deja.com/ > Before you buy.
In article <39AFF7FF.C91793B7@bloomberg.net>, kagel@bloomberg.net wrote: > If these are indeed COOKED devices and not filesystem files then all you > have to do is: > > 0) take a level 0 archive because better safe than sorry > 1) shutdown > 2) relink the chunkpaths that now point to the COOKED devices to now point > to the RAW device file with the same major/minor device numbers. > 3) restart the engine. > > Remember that the RAW and COOKED device files point to the IDENTICAL disk > space they just block it differently and cache or do not cache. Been there, > done this, it works > > Art S. Kagel > [Snip] By Art's definition, the chunks are filesystem files. Will dd be a safe alternative to migrate from filesystem files to raw devices or is Ray's suggestion - mirror the chunks, then offline/removing the primary chunk, a better way to go? Thanks for all suggestions. Sent via Deja.com http://www.deja.com/ Before you buy.
Madison Pruet <mpruet@home.com> writes: > One very important warning. > On some systems the first few K of a raw device will be overlayed by > system information. You can not simply copy the files from a cooked chunk > onto a raw device if your cooked chunk did not have an offset and your raw > space requires one. I've heard this repeated often enough, but never actually seen it. Consequently, I'm skeptical. Can someone name a system where this is indeed the case? -- Forte International, P.O. Box 1412, Ridgecrest, CA 93556-1412 Ronald Cole <ronald@forte-intl.com> Phone: (760) 499-9142 President, CEO Fax: (760) 499-9152 My GPG fingerprint: C3AF 4BE9 BEA6 F1C2 B084 4A88 8851 E6C8 69E3 B00B
I'm pretty sure that this was the case on AIX. I'm not sure if it is still the case, but a few years ago, I worked with a custom that had not used an offset. Everything looked fine untill he rebooted the system. Then the first 8K or so was overwritten by the OS. Ronald Cole wrote: > Madison Pruet <mpruet@home.com> writes: > > One very important warning. > > On some systems the first few K of a raw device will be overlayed by > > system information. You can not simply copy the files from a cooked chunk > > onto a raw device if your cooked chunk did not have an offset and your raw > > space requires one. > > I've heard this repeated often enough, but never actually seen it. > Consequently, I'm skeptical. Can someone name a system where this is > indeed the case? > > -- > Forte International, P.O. Box 1412, Ridgecrest, CA 93556-1412 > Ronald Cole <ronald@forte-intl.com> Phone: (760) 499-9142 > President, CEO Fax: (760) 499-9152 > My GPG fingerprint: C3AF 4BE9 BEA6 F1C2 B084 4A88 8851 E6C8 69E3 B00B
In article <m366oef8jr.fsf@yakisoba.forte-intl.com>, Ronald Cole <ronald@forte-intl.com> writes >Madison Pruet <mpruet@home.com> writes: >> One very important warning. >> On some systems the first few K of a raw device will be overlayed by >> system information. You can not simply copy the files from a cooked chunk >> onto a raw device if your cooked chunk did not have an offset and your raw >> space requires one. > >I've heard this repeated often enough, but never actually seen it. >Consequently, I'm skeptical. Can someone name a system where this is >indeed the case? > Solaris 2.6 using Veritas File Manager. We were setting up a new system and forgot the offset - result was that we had to recreate the chunks with correct offsets. -- Surfer!
On AIX the first 'Page' (4Kb) is used by the LVM for Logical Volume information. I have been using systems with and without a 1 page offset for years with no ill effects, but that may be good fortune. If you use command getlvcb -At <lvname> on a raw device that did not use an offset you will get gibberish. In article <RrmBwTADBhs5EwNR@nevis-view.demon.co.uk>, Surfer! <nevis- view@nospam.demon.co.uk> writes >In article <m366oef8jr.fsf@yakisoba.forte-intl.com>, Ronald Cole ><ronald@forte-intl.com> writes >>Madison Pruet <mpruet@home.com> writes: >>> One very important warning. >>> On some systems the first few K of a raw device will be overlayed by >>> system information. You can not simply copy the files from a cooked chunk >>> onto a raw device if your cooked chunk did not have an offset and your raw >>> space requires one. >> >>I've heard this repeated often enough, but never actually seen it. >>Consequently, I'm skeptical. Can someone name a system where this is >>indeed the case? >> >Solaris 2.6 using Veritas File Manager. We were setting up a new system >and forgot the offset - result was that we had to recreate the chunks >with correct offsets. > Colin Dawson
Ronald Cole wrote: > Madison Pruet <mpruet@home.com> writes: > > One very important warning. > > On some systems the first few K of a raw device will be overlayed by > > system information. You can not simply copy the files from a cooked chunk > > onto a raw device if your cooked chunk did not have an offset and your raw > > space requires one. > > I've heard this repeated often enough, but never actually seen it. > Consequently, I'm skeptical. Can someone name a system where this is > indeed the case? > > -- > Forte International, P.O. Box 1412, Ridgecrest, CA 93556-1412 > Ronald Cole <ronald@forte-intl.com> Phone: (760) 499-9142 > President, CEO Fax: (760) 499-9152 > My GPG fingerprint: C3AF 4BE9 BEA6 F1C2 B084 4A88 8851 E6C8 69E3 B00B It is true on Solaris if you are just using format, and not some fancy disk manager, or at least this was true 3 years ago. Will
cyclonejls@my-deja.com wrote: > > In article <39AFF7FF.C91793B7@bloomberg.net>, > kagel@bloomberg.net wrote: > > If these are indeed COOKED devices and not filesystem files then all > you > > have to do is: > > > > 0) take a level 0 archive because better safe than sorry > > 1) shutdown > > 2) relink the chunkpaths that now point to the COOKED devices to now > point > > to the RAW device file with the same major/minor device numbers. > > 3) restart the engine. > > > > Remember that the RAW and COOKED device files point to the IDENTICAL > disk > > space they just block it differently and cache or do not cache. Been > there, > > done this, it works > > > > Art S. Kagel > > > [Snip] > > By Art's definition, the chunks are filesystem files. Will dd be a > safe alternative to migrate from filesystem files to raw devices or is > Ray's suggestion - mirror the chunks, then offline/removing the primary > chunk, a better way to go? That will depend on whether you can afford to take the engine offline during the copy. If so then the dd will likely be faster, especially if you copy the various chunks in parallel. If you cannot afford to take the engine offline for the duration of the copy operation then the mirroring scheme is sound. Art S. Kagel
Colin Dawson <colin@firmdata-systems.co.uk> writes: > On AIX the first 'Page' (4Kb) is used by the LVM for Logical Volume > information. I have been using systems with and without a 1 page offset > for years with no ill effects, but that may be good fortune. If you use > command getlvcb -At <lvname> on a raw device that did not use an offset > you will get gibberish. Ok... That's what I expected. There is no reason that I can think of for a Unix-style kernel to put "stuff" in raw partitions. The nasty things that put "stuff" there are logical volume managers (probably figuring that "hey, if there's on fs on it, nobody must be using it"). -- Forte International, P.O. Box 1412, Ridgecrest, CA 93556-1412 Ronald Cole <ronald@forte-intl.com> Phone: (760) 499-9142 President, CEO Fax: (760) 499-9152 My GPG fingerprint: C3AF 4BE9 BEA6 F1C2 B084 4A88 8851 E6C8 69E3 B00B