RSS Appears to stall ?

Here's an odd one... We have 4 servers, all Solaris Sparc 64bit --- 2 in site A, 2 in site B Site A contains a Standalone Server S1 and RSS node S2 Site B contains RSS nodes S3 and S4 S2 and S4 stay nicely in sync with S1 ; S3 RSS applies seem to 'stall' --- the drift varies from 1 log difference to 50 logs different. All nodes are running 11.50.FC5 Restarting the engine on S3 seems to resolve the issue; however I don't want our junior DBA getting into the mindset of restarting a live engine 'being ok'. Does anyone have any ideas on obvious (or not so obvious) things we can do before we disappear down the PMR route ? Regards, Nick

Lello, Nick wrote: > Here's an odd one' > > > > We have 4 servers, all Solaris Sparc 64bit --- 2 in site A, 2 in site B > > > > Site A contains a Standalone Server S1 and RSS node S2 > > Site B contains RSS nodes S3 and S4 > > > > S2 and S4 stay nicely in sync with S1 ; S3 RSS applies seem to 'stall' > --- the drift varies from 1 log difference to 50 logs different. > > > > All nodes are running 11.50.FC5 > > > > Restarting the engine on S3 seems to resolve the issue; however I don't > want our junior DBA getting into the mindset of restarting a live engine > 'being ok'. > > > > > > Does anyone have any ideas on obvious (or not so obvious) things we can > do before we disappear down the PMR route ? > > > > Regards, > > Nick > > > Can you post onstat -g rss from both s1 and s3, and a snapshot of the logical logs current from s1 and s3? Does it catch up and slow down (i.e. keep connected but s3 lag time varies)? Seems a bit odd as s4 works okay; same sqlhosts file in use for s3 and s4. Any load on s3? noage set on both s3 and s4?

SQLHosts (there is a discrenpency in group definitions, however ER is not initialised); S1: olap_new_hdr1 ontlitcp *s1 4690 olap_rss3 ontlitcp s3 6553 olap_rss4 ontlitcp s4 6553 S3: olap_new_hdr1 ontlitcp s1 4690 g=g_olap olap_rss3 ontlitcp s3 6553 g=g_olap S4: olap_new_hdr1 ontlitcp s1 4690 g=g_olap olap_rss4 ontlitcp s4 6553 g=g_olap >From s1 [s3 data will be on the next email]:- IBM Informix Dynamic Server Version 11.50.FC5 -- On-Line -- Up 9 days 04:10:02 -- 9053184 Kbytes Local server type: Primary Index page logging status: Enabled Index page logging was enabled at: 2008/12/02 02:32:36 Number of RSS servers: 3 RSS Server information: RSS Srv RSS Srv Connection Next LPG to send Supports name status status (log id,page) Proxy Writes s2 Active Connected 166982,168 Y s3 Active Connected 166935,17910 Y s4 Active Connected 166982,96 Y onstat -l IBM Informix Dynamic Server Version 11.50.FC5 -- On-Line -- Up 9 days 04:10:56 -- 9053184 Kbytes Physical Logging Buffer bufused bufsize numpages numwrits pages/io P-2 118 192 13797682 106423 129.65 phybegin physize phypos phyused %used 17:23306684 3850000 1082102 579526 15.05 Logical Logging Buffer bufused bufsize numrecs numpages numwrits recs/pages pages/io L-3 0 96 392869368 62823592 22750428 6.3 2.8 Subsystem numrecs Log Space used OLDRSAM 392869341 101377348680 HA 27 1188 address number flags uniqid begin size used %used 2e6322f98 685 U-B---- 166773 14:2201653 51200 51200 100.00 2e6d67f50 684 U-B---- 166774 14:2150453 51200 51200 100.00 2e6d67fb8 683 U-B---- 166775 14:2099253 51200 51200 100.00 2e6d4fe50 682 U-B---- 166776 14:2048053 51200 51200 100.00 2e6d4feb8 681 U-B---- 166777 14:1996853 51200 51200 100.00 2e6d4ff20 680 U-B---- 166778 14:1894453 51200 51200 100.00 2e6d4ff88 679 U-B---- 166779 14:1945653 51200 51200 100.00 2e6323418 642 U-B---- 166780 14:53 51200 51200 100.00 2e6323480 643 U-B---- 166781 14:51253 51200 51200 100.00 2e63234e8 644 U-B---- 166782 14:102453 51200 51200 100.00 2e6323550 645 U-B---- 166783 14:153653 51200 51200 100.00 2e63235b8 646 U-B---- 166784 14:204853 51200 51200 100.00 2e6323620 647 U-B---- 166785 14:256053 51200 51200 100.00 2e6323688 648 U-B---- 166786 14:307253 51200 51200 100.00 2e63236f0 649 U-B---- 166787 14:358453 51200 51200 100.00 2e6323758 650 U-B---- 166788 14:409653 51200 51200 100.00 2e63237c0 651 U-B---- 166789 14:460853 51200 51200 100.00 2e6323828 652 U-B---- 166790 14:512053 51200 51200 100.00 2e6323890 653 U-B---- 166791 14:563253 51200 51200 100.00 2e63238f8 654 U-B---- 166792 14:614453 51200 51200 100.00 2e6323960 655 U-B---- 166793 14:665653 51200 51200 100.00 2e63239c8 656 U-B---- 166794 14:716853 51200 51200 100.00 2e6323a30 657 U-B---- 166795 14:768053 51200 51200 100.00 2e6323a98 658 U-B---- 166796 14:819253 51200 51200 100.00 2e6323b00 659 U-B---- 166797 14:870453 51200 51200 100.00 2e6323b68 660 U-B---- 166798 14:921653 51200 51200 100.00 2e6323bd0 661 U-B---- 166799 14:972853 51200 51200 100.00 2e6323c38 662 U-B---- 166800 14:1024053 51200 51200 100.00 2e6323ca0 663 U-B---- 166801 14:1075253 51200 51200 100.00 2e6323d08 664 U-B---- 166802 14:1126453 51200 51200 100.00 2e6323d70 665 U-B---- 166803 14:1177653 51200 51200 100.00 2e6323dd8 666 U-B---- 166804 14:1228853 51200 51200 100.00 2e6323e40 667 U-B---- 166805 14:1280053 51200 51200 100.00 2e6323ea8 668 U-B---- 166806 14:1331253 51200 51200 100.00 2e6323f10 669 U-B---- 166807 14:1382453 51200 51200 100.00 2e6323f78 670 U-B---- 166808 14:1433653 51200 51200 100.00 2e7d44028 671 U-B---- 166809 14:1484853 51200 51200 100.00 2e7d44090 672 U-B---- 166810 14:1536053 51200 51200 100.00 2e7d440f8 673 U-B---- 166811 14:1587253 51200 51200 100.00 2e7d44160 674 U-B---- 166812 14:1638453 51200 51200 100.00 2e7d441c8 675 U-B---- 166813 14:1689653 51200 51200 100.00 2e7d44230 676 U-B---- 166814 14:1740853 51200 51200 100.00 2e7d44298 677 U-B---- 166815 14:1792053 51200 51200 100.00 2e7d44300 678 U-B---- 166816 14:1843253 51200 51200 100.00 2e7d44368 686 U-B---- 166817 14:2252853 51200 51200 100.00 2e7d443d0 687 U-B---- 166818 14:2304053 51200 51200 100.00 2e7d44438 688 U-B---- 166819 14:2355253 51200 51200 100.00 2e7d444a0 689 U-B---- 166820 14:2406453 51200 51200 100.00 2e7d44508 690 U-B---- 166821 14:2457653 51200 51200 100.00 2e7d44570 691 U-B---- 166822 14:2508853 51200 51200 100.00 2e7d445d8 692 U-B---- 166823 14:2560053 51200 51200 100.00 2e7d44640 693 U-B---- 166824 14:2611253 51200 51200 100.00 2e7d446a8 694 U-B---- 166825 14:2662453 51200 51200 100.00 2e7d44710 695 U-B---- 166826 14:2713653 51200 51200 100.00 2e7d44778 696 U-B---- 166827 14:2764853 51200 51200 100.00 2e7d447e0 697 U-B---- 166828 14:2816053 51200 51200 100.00 2e7d44848 698 U-B---- 166829 14:2867253 51200 51200 100.00 2e7d448b0 2 U-B---- 166830 7:51253 51200 51200 100.00 2e7d44918 699 U-B---- 166831 14:2918453 51200 51200 100.00 2e7d44980 700 U-B---- 166832 14:2969653 51200 51200 100.00 2e7d449e8 701 U-B---- 166833 14:3020853 51200 51200 100.00 2e7d44a50 702 U-B---- 166834 14:3072053

>From s3:- IBM Informix Dynamic Server Version 11.50.FC5 -- Updatable (RSS) -- Up 2 days 17:52:35 -- 18714624 Kbytes RSS Server control block: 0x54596e4d0 Local server type: RSS Server Status : Active Source server name: s1 Connection status: Connected Last log page received(log id,page): 166935,16300 Sequence number of last buffer received: 1889830 Sequence number of last buffer acked: 1889816 Onstat -l IBM Informix Dynamic Server Version 11.50.FC5 -- Updatable (RSS) -- Up 2 days 17:53:16 -- 18714624 Kbytes Physical Logging Buffer bufused bufsize numpages numwrits pages/io P-2 166 192 13681948 91346 149.78 phybegin physize phypos phyused %used 17:23306684 3850000 1018299 923988 24.00 Logical Logging Buffer bufused bufsize numrecs numpages numwrits recs/pages pages/io L-1 0 96 0 0 0 0.0 0.0 Subsystem numrecs Log Space used address number flags uniqid begin size used %used 542e0ff98 685 U-B---- 166773 14:2201653 51200 51200 100.00 5454c4a28 684 U-B---- 166774 14:2150453 51200 51200 100.00 5454c4a90 683 U-B---- 166775 14:2099253 51200 51200 100.00 5454c4af8 682 U-B---- 166776 14:2048053 51200 51200 100.00 5454c4b60 681 U-B---- 166777 14:1996853 51200 51200 100.00 5454c4bc8 680 U-B---- 166778 14:1894453 51200 51200 100.00 5454c4c30 679 U-B---- 166779 14:1945653 51200 51200 100.00 5454c4c98 642 U-B---- 166780 14:53 51200 51200 100.00 5454c4d00 643 U-B---- 166781 14:51253 51200 51200 100.00 5454c4d68 644 U-B---- 166782 14:102453 51200 51200 100.00 5454c4dd0 645 U-B---- 166783 14:153653 51200 51200 100.00 5454c4e38 646 U-B---- 166784 14:204853 51200 51200 100.00 5454c4ea0 647 U-B---- 166785 14:256053 51200 51200 100.00 5454c4f08 648 U-B---- 166786 14:307253 51200 51200 100.00 5454c4f70 649 U-B---- 166787 14:358453 51200 51200 100.00 544303f50 650 U-B---- 166788 14:409653 51200 51200 100.00 544303fb8 651 U-B---- 166789 14:460853 51200 51200 100.00 5442ebf50 652 U-B---- 166790 14:512053 51200 51200 100.00 5442ebfb8 653 U-B---- 166791 14:563253 51200 51200 100.00 5456e7028 654 U-B---- 166792 14:614453 51200 51200 100.00 5456e7090 655 U-B---- 166793 14:665653 51200 51200 100.00 5456e70f8 656 U-B---- 166794 14:716853 51200 51200 100.00 5456e7160 657 U-B---- 166795 14:768053 51200 51200 100.00 5456e71c8 658 U-B---- 166796 14:819253 51200 51200 100.00 5456e7230 659 U-B---- 166797 14:870453 51200 51200 100.00 5456e7298 660 U-B---- 166798 14:921653 51200 51200 100.00 5456e7300 661 U-B---- 166799 14:972853 51200 51200 100.00 5456e7368 662 U-B---- 166800 14:1024053 51200 51200 100.00 5456e73d0 663 U-B---- 166801 14:1075253 51200 51200 100.00 5456e7438 664 U-B---- 166802 14:1126453 51200 51200 100.00 5456e74a0 665 U-B---- 166803 14:1177653 51200 51200 100.00 5456e7508 666 U-B---- 166804 14:1228853 51200 51200 100.00 5456e7570 667 U-B---- 166805 14:1280053 51200 51200 100.00 5456e75d8 668 U-B---- 166806 14:1331253 51200 51200 100.00 5456e7640 669 U-B---- 166807 14:1382453 51200 51200 100.00 5456e76a8 670 U-B---- 166808 14:1433653 51200 51200 100.00 5456e7710 671 U-B---- 166809 14:1484853 51200 51200 100.00 5456e7778 672 U-B---- 166810 14:1536053 51200 51200 100.00 5456e77e0 673 U-B---- 166811 14:1587253 51200 51200 100.00 5456e7848 674 U-B---- 166812 14:1638453 51200 51200 100.00 5456e78b0 675 U-B---- 166813 14:1689653 51200 51200 100.00 5456e7918 676 U-B---- 166814 14:1740853 51200 51200 100.00 5456e7980 677 U-B---- 166815 14:1792053 51200 51200 100.00 5456e79e8 678 U-B---- 166816 14:1843253 51200 51200 100.00 5456e7a50 686 U-B---- 166817 14:2252853 51200 51200 100.00 5456e7ab8 687 U-B---- 166818 14:2304053 51200 51200 100.00 5456e7b20 688 U-B---- 166819 14:2355253 51200 51200 100.00 5456e7b88 689 U-B---- 166820 14:2406453 51200 51200 100.00 5456e7bf0 690 U-B---- 166821 14:2457653 51200 51200 100.00 5456e7c58 691 U-B---- 166822 14:2508853 51200 51200 100.00 5456e7cc0 692 U-B---- 166823 14:2560053 51200 51200 100.00 5456e7d28 693 U-B---- 166824 14:2611253 51200 51200 100.00 5456e7d90 694 U-B---- 166825 14:2662453 51200 51200 100.00 5456e7df8 695 U-B---- 166826 14:2713653 51200 51200 100.00 5456e7e60 696 U-B---- 166827 14:2764853 51200 51200 100.00 5456e7ec8 697 U-B---- 166828 14:2816053 51200 51200 100.00 5456e7f30 698 U-B---- 166829 14:2867253 51200 51200 100.00 5456e7f98 2 U-B---- 166830 7:51253 51200 51200 100.00 5456e8028 699 U-B---- 166831 14:2918453 51200 51200 100.00 5456e8090 700 U-B---- 166832 14:2969653 51200 51200 100.00 5456e80f8 701 U-B---- 166833 14:3020853 51200 51200 100.00 5456e8160 702 U-B---- 166834 14:3072053 51200 51200 100.00 5456e81c8 703 U-B---- 166835 14:3123253 51200 51200 100.00 5456e8230 704 U-B---- 166836 14:3174453 51200 51200 100.00 5456e8298 705 U-B---- 166837 14:3225653 51200 51200 100.00 5456e8300 706 U-B---- 166838 14:3276853 51200 51200 100.00 5456e8368 707 U-B---- 166839 14:3328053 51200 51200 100.00 5456e83d0 708 U-B---- 166840 14:3379253 51200 51199 100.00 5456e8438 709 U-B---- 166841 14:3430453 51200 51200 100.00 5456e84a0 710 U-B---- 166842 14:3481653 51200 51200 100.00 5456e8508

"Lello, Nick" <Nick.Lello@nielsen.com> wrote in message news:mailman.325.1259273205.6236.informix-list@iiug.org... >From s3:- > IBM Informix Dynamic Server Version 11.50.FC5 -- Updatable (RSS) -- Up > 2 days 17:52:35 -- 18714624 Kbytes Do you think it might be because they have spelt "updateable" wrongly ...?

Captain Pedantic wrote: > "Lello, Nick" <Nick.Lello@nielsen.com> wrote in message > news:mailman.325.1259273205.6236.informix-list@iiug.org... >> From s3:- >> IBM Informix Dynamic Server Version 11.50.FC5 -- Updatable (RSS) >> -- Up 2 days 17:52:35 -- 18714624 Kbytes > > Do you think it might be because they have spelt "updateable" wrongly ...? > Possibly, although debatable. Nick, can you post the online.log from s1 from log 166843 along with the s3 online.log for the same period?