RE: storing pdf's in database
Very good point Gumby. Thanks for that. You're right. We don't want my app developer coming to me saying that the database is corrupt, because someone futzed with a file on the OS. :-) ----- Original Message ----- Subject: RE: storing pdf's in database From: "Ian Michael Gumby" >;im_gumby@hotmail.com> Date: Fri, April 11, 2008 10:50 .hmmessage P .bodyclass { margin:0px; padding:0px } body.hmmessage .bodyclass { FONT-SIZE: 10pt; FONT-FAMILY:Tahoma } Personally I'd use Python over Perl. (Python is the *new* Perl... ;-) Its smokin! and Carsten did the adapter right. (unlike cx_Oracle).... But the point that you have to consider is that if you store the .pdf outside of the database, you tend to lose control over the document. That is, even if you create a file system that is owned by Informix, and only Informix has r/w permissions, root will still have access. (Assuming Linux or Unix) So the documents can be changed, removed, etc and you'd have no auditing capabilities. (Ok, so you actually can use a third party admin tool to audit the filesystem, but that doesn't change the fact that IDS doesn't know of any changes) Because you can get extremely good performance from IDS in streaming a blob from the database, I'd recommend saving the .pdf as a blob. But then again, I'm a paranoid architect, and it depends on what .pdfs you're saving. If you took the time to make them a .pdf, I would imagine that you'd want control and be paranoid too. But what do I know? I'm saving the world from people who blindly follow their nav systems and drive in to ponds, lakes, and people's homes. ;-) -G To: informix-list@iiug.org Subject: Re: storing pdf's in database From: mvakeel@us.ibm.com Date: Fri, 11 Apr 2008 09:16:46 -0500 I have helped a customer where we use perl(DBI/DBD) to decipher e-mail attachments and upload it to smart blobs in the database, and on the front end they use the Web Datablade module to retrieve, view and manage the documents. Have had no issues with performance so far. I doubt there would be a best recommended method as such, -Manoj caver >;dmcbryde@courts.state.va.us> Sent by: informix-list-bounces@iiug.org 04/11/2008 08:10 AM To informix-list@iiug.org cc Subject Re: storing pdf's in database On Apr 10, 4:01 pm, "Floyd Wellershaus" >;fl...@fwellers.com> wrote: > Paul Thank you. > So where did the indexes get stored, in a regular table dbspace or did you > need an sbspace ? > > Thanks, > floyd > > > > > > ----- Original Message ----- > Subject: Re: storing pdf's in database > Date: Thu, April 10, 2008 15:18 > From: "Paul Watson (Oninit)" >;p...@oninit.com> > Floyd Wellershaus wrote: > > > We have IDS10.0FC5 on Aix5.3. > > > > Is there a best recommended method for storing and retrieving pdf files > > > to the database, that would be put there through an odbc connection > > > coming from powerbuilder ? > > > > I am thinking a clob or blob in an sbspace that would be accessed via > esqlc. > > > > As an alternative, is there a datablade that would allow the file to be > > > stored on the unix OS with the pointer existing in the database ? > > > > Thanks much, > > > Floyd > > > I used to use the ETX blade for this. But the actual PDFs were stored > > on the filesystem, only the indices where stored in the database. The > > ETX blade is very good on PDFs > > > _______________________________________________ > > Informix-list mailing list > > Informix-l...@iiug.org > >http://www.iiug.org/mailman/listinfo/informix-list- Hide quoted text - > > - Show quoted text - We store images as bytes in a dbspace so that we can replicate the images to another server via ER. We have fairly consistent file sizes, so we did not need smartblobs. For 4+ million images the retrieve times have never been a problem. We have a keyword index stored as another table in dbspace. That way we let informix handle all the storage and just make sure we feed the engine enough raw disk space to keep shoving in images. We use odbc, but VB (not powerbuilder) as a front end. The setup seems to work fairly well. _______________________________________________ Informix-list mailing list Informix-list@iiug.org http://www.iiug.org/mailman/listinfo/informix-list Get in touch in an instant. Get Windows Live Messenger now. _______________________________________________ Informix-list mailing list Informix-list@iiug.org http://www.iiug.org/mailman/listinfo/informix-list
If you are not indexing the data in the PDF then your database will not be corrupt - you just get 'file not found' type error. If it is indexed the you are right you can get a mismatch between the data the DB thinks it has and the file on the file system. Doesn't matter where the data is held no one is safe from an idiot with a root password. Another thing to consider is the backup - if you have 10+ M images at 100K each or a few K of PDFs at 10M each and they are essentially static data then they might as well just be on the FS and backed up once in a blue moon as opposed to every level zero. Cheers Paul Floyd Wellershaus wrote: > Very good point Gumby. Thanks for that. You're right. We don't want my > app developer coming to me saying that the database is corrupt, because > someone futzed with a file on the OS. :-) > > > > > > ----- Original Message ----- > Subject: RE: storing pdf's in database > From: "Ian Michael Gumby" <im_gumby@hotmail.com> > Date: Fri, April 11, 2008 10:50 > > > Personally I'd use Python over Perl. (Python is the *new* Perl... ;-) > Its smokin! and Carsten did the adapter right. (unlike cx_Oracle).... > > But the point that you have to consider is that if you store the .pdf > outside of the database, you tend to lose control over the document. > That is, even if you create a file system that is owned by Informix, and > only Informix has r/w permissions, root will still have access. > (Assuming Linux or Unix) So the documents can be changed, removed, etc > and you'd have no auditing capabilities. (Ok, so you actually can use a > third party admin tool to audit the filesystem, but that doesn't change > the fact that IDS doesn't know of any changes) > > Because you can get extremely good performance from IDS in streaming a > blob from the database, I'd recommend saving the .pdf as a blob. > > But then again, I'm a paranoid architect, and it depends on what .pdfs > you're saving. If you took the time to make them a .pdf, I would imagine > that you'd want control and be paranoid too. > > But what do I know? I'm saving the world from people who blindly follow > their nav systems and drive in to ponds, lakes, and people's homes. ;-) > > -G > > ------------------------------------------------------------------------ > To: informix-list@iiug.org > Subject: Re: storing pdf's in database > From: mvakeel@us.ibm.com > Date: Fri, 11 Apr 2008 09:16:46 -0500 > > > I have helped a customer where we use perl(DBI/DBD) to decipher > e-mail attachments and upload it to smart blobs in the database, > and on the front end they use the Web Datablade module to retrieve, > view and manage the documents. Have had no issues with performance > so far. I doubt there would be a best recommended method as such, > > -Manoj > > > > *caver <dmcbryde@courts.state.va.us>* > Sent by: informix-list-bounces@iiug.org 04/11/2008 08:10 AM > > To > informix-list@iiug.org > cc > > Subject > Re: storing pdf's in database > > > > > > > > > > On Apr 10, 4:01 pm, "Floyd Wellershaus" <fl...@fwellers.com> wrote: > > Paul Thank you. > > So where did the indexes get stored, in a regular table dbspace > or did you > > need an sbspace ? > > > > Thanks, > > floyd > > > > > > > > > > > > ----- Original Message ----- > > Subject: Re: storing pdf's in database > > Date: Thu, April 10, 2008 15:18 > > From: "Paul Watson (Oninit)" <p...@oninit.com> > > Floyd Wellershaus wrote: > > > > We have IDS10.0FC5 on Aix5.3. > > > > > > Is there a best recommended method for storing and retrieving > pdf files > > > > to the database, that would be put there through an odbc > connection > > > > coming from powerbuilder ? > > > > > > I am thinking a clob or blob in an sbspace that would be > accessed via > > esqlc. > > > > > > As an alternative, is there a datablade that would allow the > file to be > > > > stored on the unix OS with the pointer existing in the > database ? > > > > > > Thanks much, > > > > Floyd > > > > > I used to use the ETX blade for this. But the actual PDFs were > stored > > > on the filesystem, only the indices where stored in the > database. The > > > ETX blade is very good on PDFs > > > > > _______________________________________________ > > > Informix-list mailing list > > > Informix-l...@iiug.org > > >http://www.iiug.org/mailman/listinfo/informix-list- Hide quoted > text - > > > > - Show quoted text - > > We store images as bytes in a dbspace so that we can replicate the > images > to another server via ER. > We have fairly consistent file sizes, so we did not need smartblobs. > For 4+ million images the > retrieve times have never been a problem. We have a keyword index > stored as > another table in dbspace. That way we let informix handle all the > storage and just > make sure we feed the engine enough raw disk space to keep shoving in > images. > We use odbc, but VB (not powerbuilder) as a front end. > The setup seems to work fairly well. > > _______________________________________________ > Informix-list mailing list > Informix-list@iiug.org > http://www.iiug.org/mailman/listinfo/informix-list > > > ------------------------------------------------------------------------ > Get in touch in an instant. Get Windows Live Messenger now. > <http://www.windowslive.com/messenger/overview.html?ocid=TXT_TAGLM_WL_Refresh_getintouch_042008>; > ------------------------------------------------------------------------ > > _______________________________________________ > Informix-list mailing list > Informix-list@iiug.org <javascript:bodyCreateMail('Informix-list%40iiug.org')> > http://www.iiug.org/mailman/listinfo/informix-list >