Get logical log backups 4 to 5 times faster
Abstract
a) Fix network configuration on a TSM server (a wrong net mask even on an inactive interface can spoil the performance.)
b) Disable NetBIOS over TCP/IP on the TSM server (provided it's running on Windows.)
These activities brought us log backup in 10s instead of 45s.
Environment
IBM Informix server 11.70 on AIX 7.1 as TSM client v.7.1.8.
TSM server (actually IBM Spectrum Protect) v.8.1.12 on Windows server.
Evidence of a problem
a) Running `
dsmc query backup /smth/
` we see a significant delay after `
Node Name:
`
```
$ dsmc q b /smth/IBM Tivoli Storage Manager
Command Line Backup-Archive Client Interface
Client Version 6, Release 4, Level 0.0
Client date/time: 02.10.2023 18:15:21
(c) Copyright by IBM Corporation and other(s) 1990, 2012. All Rights Reserved.
Node Name: IDS
<< The delay happens here >>
Session established with server TSM1: Windows
```
b) Examining `
bar_act.log
` gives us another evidence, we see even 2 connections one by one, that doubles a total:
```
2023-09-23 11:18:54 30212224 25821408 Begin backup logical log 5064.
2023-09-23 11:18:54 30212224 25821408 Starting Filter /usr/bin/pigz -p 8.
<< +19 seconds >>
2023-09-23 11:19:13 36503636 30212224 Successfully connected to Storage Manager.
2023-09-23 11:19:19 36503636 30212224 The child process for the backup and restore filter is terminating with exit code 0.
<< +21 seconds >>
2023-09-23 11:19:40 30212224 25821408 Successfully connected to Storage Manager.
```
Investigation
First we go a way that proved later to be useless,
`
$ truss -d -o dsmc.truss-d dsmc q b /smth/`
```
3.3545: _erecv(5, 0x0000000110577850, 5, 0, 0x0000000000000000) (sleeping...)
3.3545: _erecv(5, 0x0000000110577850, 5, 0, 0x0000000000000000) = 5
23.6340: _erecv(5, 0x0000000110588D10, 1472, 0, 0x0000000000000000) = 1472
```
Then we run `
tcpdump
`. That revealed a fact, the server sends 6 `
Name query NBSTAT
` requests with a seconds long delay each. The requests got a reply `
Destination unreachable (Port unreachable)
`, that is why `
truss
` showed nothing during the delay.
Solution
Now the time has come to pay attention to the TSM server.
First, we noticed a wrong netmask (/8 instead of /16) on an inactive interface. Fixing that gave us 6 seconds benefit on establishing connection.
Second, we disabled `
NetBIOS over TCP/IP
`.
Voilà! The server quitted sending `NBSTAT` requests at all. That resulted in almost instant connection,
```
2023-10-02 17:21:38 32047244 33161400 Begin backup logical log 5225.
2023-10-02 17:21:38 32047244 33161400 Starting Filter /usr/bin/pigz -p 8.
2023-10-02 17:21:39 18088072 32047244 Successfully connected to Storage Manager.
2023-10-02 17:21:46 18088072 32047244 The child process for the backup and restore filter is terminating with exit code 0.
2023-10-02 17:21:48 32047244 33161400 Successfully connected to Storage Manager.
2023-10-02 17:21:48 32047244 33161400 Completed backup logical log 5225 (Storage Manager copy ID: 0 1816466202).
```
You can see 4.5 times gain in logical log backup duration.
#TSM
#NetBIOS
------------------------------
Sincerely,
Dennis
------------------------------
Hi
Thanks for sharing this.
I have not used Informix for many years, but I assume you are using the "onbar -l" command to send informix data?
Or directly at log switch?
Anyhow, I would guess that each time a log is backed up, it has to perform several steps, such as 1) connect to the TSM Server 2) negotiate encryption 3) authenticate
All these steps each time a log is being backed up?
Is there a more efficient way to perform backup and restores?
Maybee
Spictera Unified Storage one can mount IBM Storage Protect (TSM) on a path where backup lands. All backups are WORM protected and managed from the TSM server.
Eg
# mount -t spfs /backup
The connections and authentications are cached making new copies faster as the sessions are reused.
Regards Tomas
https://www.spictera.com
------------------------------
Tomas Dalebjörk
------------------------------
Hi, Nice one, sometimes the issue is below Informix in the stack! E.g. http://informix-technology.blogspot.com/2009/08/where-do-you-stop-as-dba-thoughts-about.html Regards, David. ------------------------------ David Williams ------------------------------