Re: question about compression with IConnect

Vivien Kobayashi wrote: > Is anyone aware of any way of compressing the data using IConnect or > any middleware? > > > Our main MFC developer has performed experiments by writing his own > middle layer with compression instead of connecting using IConnect. > These experiments show that IConnect is a major bottleneck. While the > native speed of the WAN links is indeed slower than the LAN, the > experiements got the WAN performance level up to an accetptable level. > However, this was only an experiment, and not pursued to develop a full > solution due to the complexity and cost of the implementation. We've > then talked to IBM/Informix but it did not go anywhere. The primary > suggestion we got is to move from HDR to ER but the complication and > the cost of ER is not preferrable at this point. I don't understand. IConnect is client/server connections. ER/HDR is server-to-server. Compression is useful mainly when the cost of compression is low compared to the saved cost of network transmission. We put it in for ER because with ER we are sending a lot of data to a lot of servers. Since the transmission is going to be large blocks (complete transactions) and we knew from testing that the savings would be significant, we added it for ER. We didn't think that we would get as much savings with HDR because we use logging compression and didn't think that we'd get good savings. For client server connections, the messages tend to be rather small, so the benefit would be even less. > > Our version of ideal solution is that the IConnect layer can be > optimized, potentially with built-in compression. However, it seems > compression is available but only among the servers participated in the > ER. Why isn't it available to the client connection? has anybody knew > or written any middle layer for compression to deal with the WAN > traffic? Any suggestion is appreciated. > > Thank you. > > -Vivien Kobayashi >

Recap from the email, >I don't understand. IConnect is client/server connections. ER/HDR is >server-to-server. What IBM was saying is if we make all servers local to the clients which would improve the clients performance. I agree this might be true, ex: instead of using 3Mbps bandwidth connecting to our server in California, UK users will definitely see a lot of speed improvement if the server is local to them with LAN speed(100/1000Mbps). >Compression is useful mainly when the cost of compression is low >compared to the saved cost of network transmission. We put it in for ER >because with ER we are sending a lot of data to a lot of servers. Since >the transmission is going to be large blocks (complete transactions) and >we knew from testing that the savings would be significant, we added it >for ER. We didn't think that we would get as much savings with HDR >because we use logging compression and didn't think that we'd get good >savings. >For client server connections, the messages tend to be rather small, so >the benefit would be even less. Well, I think that depends on the return of the queries.. the main table has a lot of columns (some clobs but mostly text, ideal for compression) and the transmission between server and client is not insignificant. We already know when the server and the clients are connected via LAN, the performance is great; but not when the network bandwidth is limited. I agree compression might be expensive for some; however, it seems we have excessive power of server which is shoe-shining most of time. So why not doing something useful, say compression/decompression. Just a thought. Thanks for your comments. -Vivien Kobayashi Madison Pruet wrote: > Vivien Kobayashi wrote: > > Is anyone aware of any way of compressing the data using IConnect or > > any middleware? > > > > > > Our main MFC developer has performed experiments by writing his own > > middle layer with compression instead of connecting using IConnect. > > These experiments show that IConnect is a major bottleneck. While the > > native speed of the WAN links is indeed slower than the LAN, the > > experiements got the WAN performance level up to an accetptable level. > > However, this was only an experiment, and not pursued to develop a full > > solution due to the complexity and cost of the implementation. We've > > then talked to IBM/Informix but it did not go anywhere. The primary > > suggestion we got is to move from HDR to ER but the complication and > > the cost of ER is not preferrable at this point. > > I don't understand. IConnect is client/server connections. ER/HDR is > server-to-server. > > Compression is useful mainly when the cost of compression is low > compared to the saved cost of network transmission. We put it in for ER > because with ER we are sending a lot of data to a lot of servers. Since > the transmission is going to be large blocks (complete transactions) and > we knew from testing that the savings would be significant, we added it > for ER. We didn't think that we would get as much savings with HDR > because we use logging compression and didn't think that we'd get good > savings. > > For client server connections, the messages tend to be rather small, so > the benefit would be even less. > > > > > Our version of ideal solution is that the IConnect layer can be > > optimized, potentially with built-in compression. However, it seems > > compression is available but only among the servers participated in the > > ER. Why isn't it available to the client connection? has anybody knew > > or written any middle layer for compression to deal with the WAN > > traffic? Any suggestion is appreciated. > > > > Thank you. > > > > -Vivien Kobayashi > >