Design and development of the application monitoring the use of server resources for server maintenance

3.1 Data collection

The first step is an interview with the head of the production department. After obtaining the data that the writer needs, the writer develops web application monitoring utilizing a waterfall timeline and Kanban technique. The rationale for adopting the Kanban method of development is that it has 14 reasons to be chosen for software development [13].

1. Panoramic perspective

2. Establishing a bottleneck

3. Self-managed teams

4. The sequence in which features are released

5. Complete focus on the critical

6. Concentration on job

7. Adaptability

8. The capacity to know everything

9. The absence of the necessity to evaluate characteristics

10. Less chit-chat and more action

11. Cohesion

12. More frequent errors

13. Increased flow

14. Concentrate on a single job.

3.2 Data analysis

Once the need is obtained, the following step is to decide which features may be implemented and which tools can be utilized. Following discussion, JavaMelody is already deployed on all client servers. JavaMelody is a free and open-source monitoring solution for JavaEE applications. JavaMelody’s purpose is to monitor Java or Java EE applications in quality assurance and production settings. It is not a tool for simulating user requests; rather, it is a tool for measuring and calculating statistics on an application’s real-world functioning based on actual use [14].

The writer then does research on other technologies for graph display in web applications and discovered Grafana’s ability to create graphs. Grafana is a free and open-source visualization and analysis tool. This enables querying, visualization, alerting, and exploration of metrics stored in any location [15]. The issue here is that Grafana cannot directly access JavaMelody, necessitating the use of an intermediary application called Prometheus.

Prometheus is a free and open source monitoring and alerting system developed by SoundCloud. Since its debut in 2012, Prometheus has been embraced by a large number of businesses and organizations. Prometheus can read JavaMelody’s API, while Grafana can read Prometheus’s data.

3.3 Present condition

At the moment, the mobile production developer is required to perform monthly visits or maintenance to check the webapp server’s health and to record the daily use of webapp server resources in the form of CPU, memory usage, active thread, transactions per minute, and garbage collector. At the moment, tracking daily webapp server resource consumption is still done manually by inspecting JavaMelody, a procedure that consumes between 1.5 and 2 h per server. Because data collection is still manual, it is prone to mistakes, and each developer has a unique interpretation of the webapp server resources graph.

3.4 Application design

Create a flowchart, an activity diagram, an entity connection diagram, and a narrative board using the data that have already been gathered. The chart shown in Figure 2 illustrates the web application Java application server monitoring flowchart.

Figure 2

Flowchart web application Java application server monitoring.

Details of explanation of storyboard for web application monitoring can be seen in Table 2.

Table 2

Storyboard

Storyboard
Project: Monitoring server	Layer: 1 of 2
Screen ID: Home
Description:
At homepage, there are several lists of webapp servers that can be clicked to open the server details page.
Link from screen ID: -	Link to screen ID: Server details
Text attribute: 1: Arial, 2.75vw, color white #ffffff, background color black #000000
3: Arial, 1.75vw, color grey #dcdde1
4: Arial, 1.75vw, color green #2ecc71
5: Arial, 1.75vw, color red #FE0101
6: Arial, 1.75vw, color yellow #FFFF00
7: 2vw, color black #000000, border 2px solid #9392AE, border radius 90px
Background: 2: color black #000000

Storyboard
Project: Monitoring server
Screen ID: Server details
Description:
At Server detail page, there is a feature to search the date between two dates to display the webapp server resources usage graph on the screen. At the top right there is a text “Back”, if clicked then web application goes back to home page
Link from screen ID: Home	Link to screen ID: -
Text attribute: 2: Arial, 2.64vw, color white #ffffff, background color black #000000
3,7: 1.5vw, color white #ffffff
4,8: input type date
5,6,9,10: input type number
11: input type submit, 1.2vw
12: 1.5vw
Background: 1: color black #000000

3.5 Application development using Kanban method

As the name implies, web application for server monitoring is developed using Kanban method. List of Kanban card and its each estimation (days) can be seen in Table 3.

Additionally, the writer used a simple waterfall chronology for the timeline as shown in Table 4. The aim of this timeline is to explain the amount of time that will be spent processing requirements from start to finish.

3.6 Submission and development

The results of black box testing for server monitoring application are shown in Table 5.

4.1 Application implementation

To perform this webapp application monitoring, we need the following hardware and software configurations: Minimum hardware specifications allow for web server monitoring that watches about 35 web apps in 15–25 s; however, we increase the timeout to 20 s if the connection is sluggish. As a result, each loop takes about 35–45 s to complete. To ensure that the program runs smoothly, each loop must be completed in less than 1 min. As a result, we need the following minimum hardware requirements as shown in Table 6.

Software definition that is both minimal and optimal. The reason Prometheus and Grafana need the right version is because the automated dashboard required to execute this application is obsolete and will be removed in the future edition of Grafana. For that reason, as shown in Table 7, we utilize Grafana v6.0.2, which retains the programmed dashboard, and then Prometheus v2.8.0, which was published at the same time as Grafana v6.0.2.

4.2 Application guidelines

The following is a tutorial for running the application. On the home page, the top part displays language such as current server monitor and server status, which includes Maintenance, Up, and Down, as well as the total number of web servers associated with each of those statuses. Each status is explained as follows:

1. The grayed-out Maintenance Status shows that the webapp server is undergoing maintenance.

2. The Up Status, which is highlighted in green, shows that the webapp server is operational and accessible to users.

3. The Down Status, which is highlighted in red, shows that the webapp server is unavailable to users. This may also occur if the web server monitoring service does not get a response from the web application server after about 20 s (timeout).

4. The Danger Status, which is highlighted in yellow, shows that the webapp server’s resource usage has exceeded the configured limit.

Figure 3 shows a list of web application servers, along with their current state. When we click on one of the servers, the system takes us to the server’s details page.

Figure 3

List of web application servers.

If there is no connection, system will not show anything but a text “There is no internet connection” as shown in Figure 4.

Figure 4

Home page with no internet connection.

When the user clicks on one of the server’s home page, the server details page will appear as shown in Figure 5. It includes the webapp’s name in the upper middle area. The top right corner has a back button that takes us back to the main page. It includes an input area for the time filter underneath the name section.

Figure 5

Time filter period input example.

Additionally, this page includes six graphs that illustrate the webapp server’s resource consumption. The following is an explanation of the six graphs.

1. Active Threads. These are used to refer to the processes that are now being executed on the webapp server.

2. System CPU. The system CPU statistic indicates the server’s CPU utilization in terms of percentage. The increased use suggests that we either need to optimize our code and queries to reduce CPU usage, or we need to update our hardware.

3. Musculoskeletal Memory. This is used to indicate the Operating System’s (OS) memory consumption.

4. Utilized Memory. This refers to the amount of Random Access Memory (RAM) used by the server.

5. Waste Disposal. This is a procedure that is used to eliminate unwanted objects. This is part of the garbage collector that is a component of the Java programming language that runs automatically until the application is terminated. This graph is necessary to determine if there is a memory leak, which may result in an error out of memory.

6. Free Drive Space. The term “Free Disk Space” refers to the leftover unused space on the server’s hard disk. The lower the graph becomes, the more likely it is that we need to delete files that are no longer needed, such as logs, or transfer the file to another disk.

We may display graph data for a certain time period by entering the date, hour, and minutes in the From and End sections and then clicking the Apply button. Here is an example of a graph over a 30 min period as shown on Figure 6.

Figure 6

Server details page with a range of 30 min.

4.3 Metrics page (API)

This page provides data about metrics in order for the Prometheus to access the API and create a graph that the Prometheus can read.

While the system CPU and garbage collector are measured in percentages, utilized memory, physical memory, and free disk space are measured in bytes. When data are received, it will measure using the current thread’s real use thread. Grafana will utilize the Prometheus API to generate a graph that can be shown in web app server monitoring once the Prometheus app creates the graph.

Addition of a List Server

If the user wishes to add or delete a web application server, they may modify the JSON data that the app can access. This approach is preferred since it is more comprehensible and adaptable.

To add or delete a server from the home page, the user may modify the object data in the JSON file using the following syntax.

{

“Server”: {

   “name”: “adimobile”,

    “url”: “https://www.xxx.com/monitoring”,

    “servertype”: “webapp”,

    “username”: “username”,

    “password”: “password”,

    “sqlmean”: “6500”,

    “sqlmax”: “7500”,

    “alertThreads”: “10”,

    “alertFreedisk”: “150000000000”,

    “alertGcmax”: “5”,

    “alertGcmean”: “3”,

    “alertSystemCPU”: “50”,

    “alertPhysicalMemory”: “16000000000”,

    “alerttime”: “5”,

    “alertDowntime”: “120”,

    “maintenancestart”: “00.00”,

    “maintenanceend”: “00.00”

   }

}

Explanation of the above JSON.

1. Name

   Name that will be displayed on the webapp server monitor.

2. Url

   App Url that will be monitored and has JavaMelody applied.

3. Servertype

   Can be choose between webapp/services

4. Username

   Username to access JavaMelody from the given url.

5. Password

   Password to access JavaMelody from the given url.

6. Sqlmean

   Average time (in seconds), which if there is a SQL query that runs above the limit of the sqlmean, then the system will record it in the database.

7. Sqlmax

   Max time (in seconds), which if there is a SQL query that runs above the sqlmax, then the system will record it in the database.

8. AlertThreads

   Maximum thread in which if there is thread usage that exceeds the limit of activeThreads, then the webapp status will change color to yellow or danger.

9. AlertFreedisk

   If the current free disk (in bytes) is below the AlertFreeDiskValue, then the webapp status will change color to yellow or danger.

10. AlertGcmax

    Maximum GC (in percentage). If GC usage volume exceeds alertGcmax, then the webapp status will change color to yellow or danger.

11. AlertGcmean

    Average GC (in percentage), if GC average usage exceeds alertGcmean value, then the webapp status will change color to yellow or danger.

12. AlertSystemCPU

    Maximum SystemCPU (in percentage), if SystemCPU usage exceeds alertSystemCPU value, then the web app status will change color to yellow or danger.

13. AlertPhysicalMemory

    Maximum PhysicalMemory (in bytes), if PhysicalMemory usage exceeds alertPhysicalMemory value, then the webapp status will change color to yellow or danger.

14. Alerttime

    if there is a webapp server that exceeds n number times alert in a row, then the system will send notification/email to the production team to check the server condition later.

15. AlertDowntime

    If email has already been sent to the receiver, then even if it exceeds n number times alert, the email will not be sent, except if the time period from the last email sent to current time has exceeded AlertDownTime.

    This will minimize spam email to the team if the server is continuously down and exceed alert limit

16. Maintenancestart

    When the maintenance will start, if the current time is in the range of maintenancestart and maintenanceend, then the webapp status will change color to gray or maintenance.

17. Maintenanceend

When the maintenance will end, if the current time is in the range of maintenancestart and maintenanceend, then the webapp status will change color to gray or maintenance.

4.4 Email alert

This feature is used to notify the production team if there is a server which has status of down or danger for n times in a row.

1. Server Down Alert

   Email will be sent to the recipient when the webapp server cannot be accessed by server monitoring or experienced timeout as shown in Figure 7.

2. Query Alert

   Email will be sent to the recipient when there is a query that exceeds mean or max time as shown in Figure 8.

3. Alert Free Disk Space email will be sent to the recipient when the free disk space is below the settled parameter. Alert GC Mean, GC Max, Physical Memory, System CPU, and Threads have the same format as Alert Free Disk Space as shown in Figure 9.

Figure 7

Alert server down example.

Figure 8

Alert query example.

Figure 9

Alert free disk space email example.

Information

Time : 00:38 GMT + 7

Metrics : 76.79GB

Limit : 150GB

4.5 Database

In the database, there are a total of 6 tables used. Name and usage of each table is as follows.

1. Email

   • Table is used to send email to recipients if there is a webapp server that experiences down or resource usage that exceeds the limit. This table has 4 columns. which is:

     1. Emailid

        Emailid is the primary key for the table.

     2. Email_to

        Email_to is the email of the recipient. If there is more than 1 recipient, we can use semi colon (;) as a separator. For example if we want to send email to two recipient: aaa@aaa.com;bbb@bbbb.com

     3. Email_cc

        Email_cc is the email of the recipient that will be given carbon copy. If there is more than 1 recipient, we can use semi colon (;) as a separator. For example if we want to send email to two recipient: aaa@aaa.com;bbb@bbbb.com

     4. Is_active

   To mark if the email_to or email_cc is still active, if not, email will not be sent.

2. Path data

   PathData table has 2 columns, which is:

   1. Directory

      Location in which the file will be read.

   2. FileType

   There are 2 rows for this table containing JSON and img

   a. JSON: Path where JSON file is located.

   b. Img: Path where image is used for attachment in the email.

3. Query data

   This table contains 8 columns as follows:

   1. QueryID (PK)

   2. ClientCode

   3. URL

   4. QueryString: Query that is running in the database.

   5. MeanTime: Average time needed to execute above QueryString.

   6. MaxTime: Maximum time needed to execute QueryString above.

   7. FlagQuery: Can be 0 or 1. 0 means there is no query LIKE.

   8. TimeStamp: Exact time when QueryString taken.

4. RV_ReportV2

   On Table RV_ReportV2 there are 7 columns:

   1. ID

   2. ClientCod

   3. Max_Memory

   4. Max_Thread

   5. Max_TPM

   6. Time_Stamp

   7. Max_GC

5. RV_ReportSummary

   RV_ReportSummary Table contains daily summary of resource usage from one webapp server that is created by stored procedure. There are eight columns of this table which is time data collected, name of server, avg memory, max memory, max thread, avg TPM, max TPM, and max GC on that day.

6. Utilities

   In Utilities Table, there are four columns as follows:

   1. ID

      Primary key of Utilities Table.

   2. Name

      There are 7 data for this column which is:

      1. Sql_start_check: start hour to check SQL query.

      2. Sql_pause_check: interval in hour between each check.

      3. Sql_end_check: end hour to check SQL query.

      4. Timeout: ping duration to the webapp server (ms)

      5. Send_sql_email: 1 or 0 (1 means email will be sent if there is SQL query that exceeds the limit)

      6. Send_email_down: 1 or 0 (1 means email will be sent if there is webapp server which experiences down time)

      7. Alert_count_threshold: number of times webapp server down is required before sending email.

   3. Value

      Value of the name. To see the example of the value, one can refer to the name section above.

   4. Description

      Description of the name utilities. To see the example of the value, one can refer to the name section above.

4.6 Comparison evaluation between JavaMelody and webapp server monitoring

Table 8 shows the comparison data between JavaMelody that is taken manually every day and web server monitoring for usage memory in Megabytes and percentage. For percentage we can use the formula as follows:

1. If Manual ≤ Webapp monitoring, then:

   Percentage = Manual Webapp monitoring × 100 %

2. If Webapp Monitoring < Manual, then: