Capturing Your Own Solaris 11.4 (12) Analytics / Sstore – Part 2

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Capturing Solaris 11.4/(12) analytical data

Note: There is an updated process for Solaris 11.4+, please check out part 5 – Enhanced method on how to capture analytics in Solaris 11.4+

Note: The full code is available in a GitHub repository. to play with the code, feel free to just clone the Solaris Analytics Publisher repository.

In the last post we prepared/configured Stats Store(sstore) with our own schema.below I am going to show you haw to capture and prepare the data to be able to populate in the Stats Store(sstore) format.

  • Please check out part 1 on how to configure analytics
  • part 2 on how to configure the client capture stat process
  • Please check out part 3 on how to publish the client captured stats.
  • Please check out part 4 Configuring / Accessing the Web Dashboard / UI.
  • Please check out part 5 Capturing Solaris 11.4 (12) Analytics By Using Remote Administration Daemon (RAD).

To see how you can populate the Stats Store(sstore) with custom data, please check out part3 of this series.

Before we can populate the sstore with data, we first need to capture the data from our sources..

Configuring the clients to capture data

There are two parts to the client configuration.

  1. The client capturing stats i.e. CPU, Memory, db, etc…
  2. The client making these stats available (remotely)

First I will show you how to collect OS stats

To capturing OS stats, I use a Python module called psutil, you can use any other form for data collection as outlined below..
Tip: Psutil is included in Solaris 11.4(12) but not in Solaris 11.

Lets install psutil (or update to latest in Solaris 11.4/12)

For Solaris 12, you can install psutil from the repo, just run the below.

Note: The below steps are for Solaris 11
If you dont have Internet access you cat download the psutil-4.x.tar.gz file from the Python modules web site.

Installing the Python module

Prerequisite required for the module to properly install.
You will need the system header package, install by running the below..

Note: In Solaris 11 you will also need cc, Download from Oracle CC (Oracle/Sun Studio).

Installing psutil

Next lets create the client capture program

Create directory

I will explain each directory content

* /opt/sys_monitor/conf – contains DB related configuration scripts
* /opt/sys_monitor/bin – python code to capture and exposed stats
* /opt/sys_monitor/db – contains local sqlite db with latest stat record
* /opt/sys_monitor/services – contains Solaris xml service files
* /opt/sys_monitor/startup – smf startup helper scripts
* /opt/sys_monitor/modules – psutil module (only needed for install)
* /opt/sys_monitor/statsSrc – contains the Stats Store Json files

DB related scripts are below

chk_db-ses.sh – get db sessions

The below script will return the number of active system sessions in MySQL which will then be populate to the Stats store(sstore) with the results.
cat /opt/sys_monitor/conf/chk_db1_apps-ses.sh

checks db query timing

The below script will run a particular query which takes some time to return (you can replace with your own query). We will then populate the Stats store with the time it took to complete the query.
cat /opt/sys_monitor/conf/test_db1_apps.sh

Python capture scripts.

Stats python capture script

The below Python capture script captures OS data as well as db data (like the two scripts above).
cat /opt/sys_monitor/bin/capture.py

Note: The above script will capture in parallel all stats, i.e. even if one stat takes longer to complete.

Stats publish script (listen socket), for remote use

The below script fetches the results from the Sqlite DB, and the SQlite DB keeps on getting updated by the above script.

The fetch script is below, it will also create the SQlite tables.
/opt/sys_monitor/bin/get_results.py

Local SQLite DB schema files

Note: the DB will auto get created at run time.

DB1 schema file

cat /opt/sys_monitor/db/db1_qry-sql_schema.sql

General monitor schema file

cat /opt/sys_monitor/db/sql_schema.sql

Create SMF startup helper scripts

Capture startup script

cat /opt/sys_monitor/startup/capture_startup.sh

publish results startup script

cat /opt/sys_monitor/startup/get_results.sh

Create SMF services

Capture service

cat /opt/sys_monitor/services/capture_service.xml

publish results

cat /opt/sys_monitor/services/getresults_service.xml

Now we are ready to start capturing

Lets import the services

Verify all is working

Example of output is below

Note: in many cases you need to update the below seeings in the capture scripts – /opt/sys_monitor/bin/capture.py
Your device can be seen with iostat -xc
Your network device can be seen with ipadm or dladm

Next, part 3, I will show you how to publish the stats in analytics, Click here to go to part 3.

You might also like – Articles related to Oracle Solaris 11.4/Solaris 12.

Like what you’re reading? please provide feedback, any feedback is appreciated.

Leave a Reply

Be the First to Comment!

avatar
  Subscribe  
Notify of