Contents 1 To add/remove CPU on a ruining zone (dedicated-cpu) 1.1 First get the pset name 1.2 Before the change just 4 cpu are configured 1.3 To add/modify the numbers of CPU to 8 1.4 After the change 2 To add/modify CPU on a ruining zone (capped-cpu) 2.1 Before the change 2.2 To make the […]
Tag: Performance
Tuning Java Garbage Collection(GC)
Tuning java and GC The link below describes in great detail how to troubleshot, tune Java Garbage Collector(GC). Garbage First Garbage Collector Tuning
Linux, Solaris, Docker, Zones – CPU hog load test
A controlled CPU HOG load test – Change the number to the number of CPU's
1 2 3 4 5 6 7 8 9 10 11 12 13 |
#!/usr/bin/perl print "Eating the CPUs\n"; foreach $i (1..16) { $pid = fork(); last if $pid == 0; print "Created PID $pid\n"; } while (1) { $x++; } |
Another form of loading all cpu’s, might not work in latest Linux bash releases Note: Make sure you are not running anything important on the system (docker or zone), as it will max all cpus.
1 |
:(){:|:&};: |
How To Measure Network Latency With Ping And Dtrace in Solaris.
Measuring Network Latency In Solaris Use the below dtrace script (SuperPing.d) to measuring Network Latency.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
#!/usr/sbin/dtrace -s #pragma D option quiet #pragma D option switchrate=10hz mib:::rawipOutDatagrams /pid == $target/ { start = timestamp; } mib:::icmpInEchoReps /start/ { this->delta = (timestamp - start) / 1000; printf("dtrace measured: %d us\n", this->delta); @a["\n ICMP packet delta average (us):"] = avg(this->delta); @q["\n ICMP packet delta distribution (us):"] = lquantize(this->delta, 0, 1000000, 100); start = 0; } |
Run it like the below.
1 2 3 |
./superping.d -c "ping -s 10.151.19.10" ctr ^c - to stop |
Sample output.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
----10.151.19.10 PING Statistics---- 62 packets transmitted, 62 packets received, 0% packet loss dtrace measured: 410 us ICMP packet delta average (us): 430 ICMP packet delta distribution (us): value ------------- Distribution ------------- count 200 | 0 300 |@@@@@@@@ 12 400 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 48 500 |@ 1 600 |@ 1 700 | 0 round-trip (ms) min/avg/max/stddev = 0.748/0.850/1.076/0.049 |
Performance Tuning For Solaris Zones
Contents 1 Solaris global and non zones performance tuning 1.1 Configuring the global zone to use FSS 1.1.1 First enable & set FSS in the global zone 1.1.2 Next configure a global zone CPU/Memory Cap 1.1.2.1 Example after the configuration change 1.2 Configure /tmp(tmpfs) system cap 1.3 reboot the system for changes to take effect […]
Capturing Linux network statistics – perl script
Capture Linux network statistics Example capture output is below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
collisions : 0 0 0 multicast : 2101 0 0 rx_bytes : 1079497308853 15334 12768 rx_compressed : 0 0 0 rx_crc_errors : 0 0 0 rx_dropped : 0 0 0 rx_errors : 0 0 0 rx_fifo_errors : 0 0 0 rx_frame_errors : 0 0 0 rx_length_errors : 0 0 0 rx_missed_errors : 897 0 0 rx_over_errors : 0 0 0 rx_packets : 1921006922 175 135 tx_aborted_errors : 0 0 0 tx_bytes : 2413670939512 22024 21054 tx_carrier_errors : 0 0 0 tx_compressed : 0 0 0 tx_dropped : 0 0 0 tx_errors : 0 0 0 tx_fifo_errors : 0 0 0 tx_heartbeat_errors : 0 0 0 tx_packets : 2450565916 146 114 tx_window_errors : 0 0 0 eth0 : 00:11:17.002477 1s 5s 15s 60s |
Perl script source is below
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
#!/usr/bin/perl use strict; use warnings; use POSIX qw(strftime); use Time::HiRes qw(gettimeofday usleep); my $dev = @ARGV ? shift : 'eth0'; my $dir = "/sys/class/net/$dev/statistics"; my %stats = do { opendir +(my $dh), $dir; local @_ = readdir $dh; closedir $dh; map +($_, []), grep !/^\.\.?$/, @_; }; if (-t STDOUT) { while (1) { print "\033[H\033[J", run(); my ($time, $us) = gettimeofday(); my ($sec, $min, $hour) = localtime $time; { local $| = 1; printf '%-31.31s: %02d:%02d:%02d.%06d%8s%8s%8s%8s', $dev, $hour, $min, $sec, $us, qw(1s 5s 15s 60s) } usleep($us ? 1000000 - $us : 1000000); } } else {print run()} sub run { map { chomp (my ($stat) = slurp("$dir/$_")); my $line = sprintf '%-31.31s:%16.16s', $_, $stat; $line .= sprintf '%8.8s', int (($stat - $stats{$_}->[0]) / 1) if @{$stats{$_}} > 0; $line .= sprintf '%8.8s', int (($stat - $stats{$_}->[4]) / 5) if @{$stats{$_}} > 4; $line .= sprintf '%8.8s', int (($stat - $stats{$_}->[14]) / 15) if @{$stats{$_}} > 14; $line .= sprintf '%8.8s', int (($stat - $stats{$_}->[59]) / 60) if @{$stats{$_}} > 59; unshift @{$stats{$_}}, $stat; pop @{$stats{$_}} if @{$stats{$_}} > 60; "$line\n"; } sort keys %stats; } sub slurp { local @ARGV = @_; local @_ = <>; @_; } |
How To Change The Priority In Solaris For The Oracle DB Process
Modify the Oracle Database Cluster process to use the FX scheduler.
1 |
priocntl -s -c FX -m 60 -p 60 -i pid `pgrep -f ora_lgwr_TST` |
Update to include the LMS process as well
1 |
for i in `pgrep -f ora_lms && pgrep -f ora_lgwr_TST`; do priocntl -s -c FX -m 60 -p 60 -i pid $i; done |
Solaris network statistics utility (only Solaris 10)
Capturing Network statistics on Solaris 10 Latest nicstat utility is available here. The latest nicstat utility is compiled and might work better, below is the older perl version. The latest version is also on sourceforge Example script output
1 2 3 4 5 |
Time Int rKB/s wKB/s rPk/s wPk/s rAvs wAvs  %Util Sat 00:26:33 aggr1 60562.9 1521.4 41090.4 20494.1 1509.3 76.02 50.86 0.00 00:26:33 nxge3 0.47 0.29 5.23 3.17 92.39 93.85 0.00 0.00 00:26:33 nxge7 0.47 0.29 5.23 3.15 92.61 93.41 0.00 0.00 00:26:33 lo0 0.00 0.00 1883.0 1883.0 0.00 0.00 0.00 0.00 |
Perl script
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 |
#!/usr/bin/perl -w # # nicstat - print network traffic, Kbyte/s read and written. # Solaris 8+, Perl (Sun::Solaris::Kstat). # # "netstat -i" only gives a packet count, this program gives Kbytes. # # 30-Sep-2006, ver 1.00 (check for new versions, http://www.brendangregg.com) # # USAGE: nicstat [-hsz] [-i int[,int...]] | [interval [count]] # # -h # help # -s # print summary output # -z # skip zero lines # -i int[,int...] # print these instances only # eg, # nicstat # print summary since boot # nicstat 1 # print continually, every 1 second # nicstat 1 5 # print 5 times, every 1 second # nicstat -i hme0 # only examine hme0 # # This prints out the KB/s transferred for all the network cards (NICs), # including packet counts and average sizes. The first line is the summary # data since boot. # # FIELDS: # Int Interface # rKB/s read Kbytes/s # wKB/s write Kbytes/s # rPk/s read Packets/s # wPk/s write Packets/s # rAvs read Average size, bytes # wAvs write Average size, bytes # %Util %Utilisation (r+w/ifspeed) # Sat Saturation (defer, nocanput, norecvbuf, noxmtbuf) # # NOTES: # # - Some unusual network cards may not provide all the details to Kstat, # (or provide different symbols). Check for newer versions of this program, # and the @Network array in the code below. # - Utilisation is based on bytes transferred divided by speed of the interface # (if the speed is known). It should be impossible to reach 100% as there # are overheads due to bus negotiation and timing. # - Loopback interfaces may only provide packet counts (if anything), and so # bytes and %util will always be zero. Newer versions of Solaris (newer than # Solaris 10 6/06) may provide loopback byte stats. # - Saturation is determined by counting read and write errors caused by the # interface running at saturation. This approach is not ideal, and the value # reported is often lower than it should be (eg, 0.0). Reading the rKB/s and # wKB/s fields may be more useful. # # SEE ALSO: # nicstat.c # the C version, also on my website # kstat -n hme0 [interval [count]] # or qfe0, ... # netstat -iI hme0 [interval [count]] # se netstat.se [interval] # SE Toolkit # se nx.se [interval] # SE Toolkit # # COPYRIGHT: Copyright (c) 2006 Brendan Gregg. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software Foundation, # Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. # # (http://www.gnu.org/copyleft/gpl.html) # # Author: Brendan Gregg [Sydney, Australia] # # 18-Jul-2004 Brendan Gregg Created this. # 07-Jan-2005 " " added saturation value. # 07-Jan-2005 " " added summary style (from Peter Tribble). # 23-Jan-2006 " " Tweaked style. # 11-Aug-2006 " " Improved output neatness. # 30-Sep-2006 " " Added loopback, tweaked output. use strict; use Getopt::Std; use Sun::Solaris::Kstat; my $Kstat = Sun::Solaris::Kstat->new(); # # Process command line args # usage() if defined $ARGV[0] and $ARGV[0] eq "--help"; getopts('hi:sz') or usage(); usage() if defined $main::opt_h; my $STYLE = defined $main::opt_s ? $main::opt_s : 0; my $SKIPZERO = defined $main::opt_z ? $main::opt_z : 0; # process [interval [count]], my ($interval, $loop_max); if (defined $ARGV[0]) { $interval = $ARGV[0]; $loop_max = defined $ARGV[1] ? $ARGV[1] : 2**32; usage() if $interval == 0; } else { $interval = 1; $loop_max = 1; } # check for -i, my %NetworkOnly; # network interfaces to print my $NETWORKONLY = 0; # match on network interfaces if (defined $main::opt_i) { foreach my $net (split /,/, $main::opt_i) { $NetworkOnly{$net} = 1; } $NETWORKONLY = 1; } # globals, my $loop = 0; # current loop number my $PAGESIZE = 20; # max lines per header my $line = $PAGESIZE; # counter for lines printed my %NetworkNames; # Kstat network interfaces my %NetworkData; # network interface data my %NetworkDataOld; # network interface data $main::opt_h = 0; $| = 1; # autoflush ### Determine network interfaces unless (find_nets()) { if ($NETWORKONLY) { print STDERR "ERROR1: $main::opt_i matched no network interfaces.\n"; } else { print STDERR "ERROR1: No network interfaces found!\n"; } exit 1; } # # Main # while (1) { ### Print Header if ($line >= $PAGESIZE) { if ($STYLE == 0) { printf "%8s %7s %7s %7s %7s %7s %7s %7s %7s %6s\n", "Time", "Int", "rKB/s", "wKB/s", "rPk/s", "wPk/s", "rAvs", "wAvs", "%Util", "Sat"; } elsif ($STYLE == 1) { printf "%8s %8s %14s %14s\n", "Time", "Int", "rKB/s", "wKB/s"; } $line = 0; } ### Get new data my (@NetworkData) = fetch_net_data(); foreach my $network_data (@NetworkData) { ### Extract values my ($int, $rbytes, $wbytes, $rpackets, $wpackets, $speed, $sat, $time) = split /:/, $network_data; ### Retrieve old values my ($old_rbytes, $old_wbytes, $old_rpackets, $old_wpackets, $old_sat, $old_time); if (defined $NetworkDataOld{$int}) { ($old_rbytes, $old_wbytes, $old_rpackets, $old_wpackets, $old_sat, $old_time) = split /:/, $NetworkDataOld{$int}; } else { $old_rbytes = $old_wbytes = $old_rpackets = $old_wpackets = $old_sat = $old_time = 0; } # # Calculate statistics # # delta time my $tdiff = $time - $old_time; # per second values my $rbps = ($rbytes - $old_rbytes) / $tdiff; my $wbps = ($wbytes - $old_wbytes) / $tdiff; my $rkps = $rbps / 1024; my $wkps = $wbps / 1024; my $rpps = ($rpackets - $old_rpackets) / $tdiff; my $wpps = ($wpackets - $old_wpackets) / $tdiff; my $ravs = $rpps > 0 ? $rbps / $rpps : 0; my $wavs = $wpps > 0 ? $wbps / $wpps : 0; # skip zero lines if asked next if $SKIPZERO and ($rbps + $wbps) == 0; # % utilisation my $util; if ($speed > 0) { # the following has a mysterious "800", it is 100 # for the % conversion, and 8 for bytes2bits. $util = ($rbps + $wbps) * 800 / $speed; $util = 100 if $util > 100; } else { $util = 0; } # saturation per sec my $sats = ($sat - $old_sat) / $tdiff; # # Print statistics # if ($rbps ne "") { my @Time = localtime(); if ($STYLE == 0) { printf "%02d:%02d:%02d %7s ", $Time[2], $Time[1], $Time[0], $int; print_neat($rkps); print_neat($wkps); print_neat($rpps); print_neat($wpps); print_neat($ravs); print_neat($wavs); printf "%7.2f %6.2f\n", $util, $sats; } elsif ($STYLE == 1) { printf "%02d:%02d:%02d %8s %14.3f %14.3f\n", $Time[2], $Time[1], $Time[0], $int, $rkps, $wkps; } $line++; # for multiple interfaces, always print the header $line += $PAGESIZE if @NetworkData > 1; } ### Store old values $NetworkDataOld{$int} = "$rbytes:$wbytes:$rpackets:$wpackets:$sat:$time"; } ### Check for end last if ++$loop == $loop_max; ### Interval sleep $interval; } # find_nets - walk Kstat to discover network interfaces. # # This walks %Kstat and populates a %NetworkNames with discovered # network interfaces. # sub find_nets { my $found = 0; ### Loop over all Kstat modules foreach my $module (keys %$Kstat) { my $Modules = $Kstat->{$module}; foreach my $instance (keys %$Modules) { my $Instances = $Modules->{$instance}; foreach my $name (keys %$Instances) { ### Skip interface if asked if ($NETWORKONLY) { next unless $NetworkOnly{$name}; } ### Skip if not the regular statistic set next unless $name =~ /^$module/; my $Names = $Instances->{$name}; # Check this is a network device. # Matching on ifspeed has been more reliable than "class" # we also match loopback interfaces. if (defined $$Names{ifspeed} || $module eq "lo") { ### Save network interface $NetworkNames{$name} = $Names; $found++; } } } } return $found; } # fetch - fetch Kstat data for the network interfaces. # # This uses the interfaces in %NetworkNames and returns useful Kstat data. # The Kstat values used are rbytes64, obytes64, ipackets64, opackets64 # (or the 32 bit versions if the 64 bit values are not there). # sub fetch_net_data { my ($rbytes, $wbytes, $rpackets, $wpackets, $speed, $time); my @NetworkData = (); $Kstat->update(); ### Loop over previously found network interfaces foreach my $name (keys %NetworkNames) { my $Names = $NetworkNames{$name}; if (defined $$Names{opackets}) { ### Fetch write bytes if (defined $$Names{obytes64}) { $rbytes = $$Names{rbytes64}; $wbytes = $$Names{obytes64}; } elsif (defined $$Names{obytes}) { $rbytes = $$Names{rbytes}; $wbytes = $$Names{obytes}; } else { $rbytes = $wbytes = 0; } ### Fetch read bytes if (defined $$Names{opackets64}) { $rpackets = $$Names{ipackets64}; $wpackets = $$Names{opackets64}; } else { $rpackets = $$Names{ipackets}; $wpackets = $$Names{opackets}; } ### Fetch interface speed if (defined $$Names{ifspeed}) { $speed = $$Names{ifspeed}; } else { # if we can't fetch the speed, print the # %Util as 0.0 . To do this we, $speed = 2 ** 48; } ### Determine saturation value my $sat = 0; if (defined $$Names{nocanput} or defined $$Names{norcvbuf}) { $sat += defined $$Names{defer} ? $$Names{defer} : 0; $sat += defined $$Names{nocanput} ? $$Names{nocanput} : 0; $sat += defined $$Names{norcvbuf} ? $$Names{norcvbuf} : 0; $sat += defined $$Names{noxmtbuf} ? $$Names{noxmtbuf} : 0; } ### use the last snaptime value, $time = $$Names{snaptime}; ### store data push @NetworkData, "$name:$rbytes:$wbytes:" . "$rpackets:$wpackets:$speed:$sat:$time"; } } return @NetworkData; } # print_neat - print a float with decimal places if appropriate. # # This specifically keeps the width to 7 characters, if possible, plus # a trailing space. # sub print_neat { my $num = shift; if ($num >= 100000) { printf "%7d ", $num; } elsif ($num >= 100) { printf "%7.1f ", $num; } else { printf "%7.2f ", $num; } } # usage - print usage and exit. # sub usage { print STDERR <<END; USAGE: nicstat [-hsz] [-i int[,int...]] | [interval [count]] eg, nicstat # print summary since boot nicstat 1 # print continually every 1 second nicstat 1 5 # print 5 times, every 1 second nicstat -s # summary output nicstat -i hme0 # print hme0 only END exit 1; } |
Solaris Dynamic Pool Resource Capping With Examples
Contents 1 Resource Capping available now 1.1 CPU capping for zones 1.2 Memory capping for zones 1.3 Memory capping settings coming soon 1.4 Resource capping Helpful Tips Resource Capping available now Note: It is now much simpler to configure caps in the most recent Solaris versions click here for an example This document assumes your […]
Script to get the Solaris ZFS ARC Size
This script will list the ZFS adaptive replacement cache (ARC)statistics and usages To get the script click here Another great ZFS ARC summery script is available here Output will look something like this.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
[root@server] /tmp # ./arc_summary.pl System Memory: Physical RAM: 64128 MB Free Memory : 58808 MB LotsFree: 999 MB ZFS Tunables (/etc/system): ARC Size: Current Size: 1854 MB (arcsize) Target Size (Adaptive): 63104 MB (c) Min Size (Hard Limit): 7888 MB (zfs_arc_min) Max Size (Hard Limit): 63104 MB (zfs_arc_max) ARC Size Breakdown: Most Recently Used Cache Size: 50% 31679 MB (p) Most Frequently Used Cache Size: 49% 31424 MB (c-p) ARC Efficency: Cache Access Total: 4158593 Cache Hit Ratio: 99% 4129443 [Defined State for buffer] Cache Miss Ratio: 0% 29150 [Undefined State for Buffer] REAL Hit Ratio: 96% 4027900 [MRU/MFU Hits Only] Data Demand Efficiency: 99% Data Prefetch Efficiency: 73% CACHE HITS BY CACHE LIST: Anon: 2% 99429 [ New Customer, First Cache Hit ] Most Recently Used: 8% 370014 (mru) [ Return Customer ] Most Frequently Used: 88% 3657886 (mfu) [ Frequent Customer ] Most Recently Used Ghost: 0% 1810 (mru_ghost) [ Return Customer Evicted, Now Back ] Most Frequently Used Ghost: 0% 304 (mfu_ghost) [ Frequent Customer Evicted, Now Back ] CACHE HITS BY DATA TYPE: Demand Data: 85% 3524673 Prefetch Data: 0% 15914 Demand Metadata: 11% 494183 Prefetch Metadata: 2% 94673 CACHE MISSES BY DATA TYPE: Demand Data: 24% 7043 Prefetch Data: 20% 5859 Demand Metadata: 39% 11613 Prefetch Metadata: 15% 4635 --------------------------------------------- |
To change the arc size, add the following line to /etc/system, where the numeric value is the desired arc size in […]