uberAgent and Elastic Stack - Getting Started

Introduction

Elastic is a company with various products. In this article, we’ll focus on Elasticsearch and Kibana, or as it’s called together Elastic Stack.
We’ll focus on the Elastic Stack with the integration of uberAgent and their use case for a Citrix Virtual Apps and Desktop (CVAD) environment.

This article will not explain Elastic in detail (and the good thing, it isn’t necessary to be an expert to use the system). I’ll give you a recommended list to read and after that a quick overview of the system.

Study materials ELK

My first recommendation is the Beginner’s Crash Course to Elastic Stack by Lisa Jung, especially the first video Intro to Elasticsearch and Kibana is very useful for our journey.

The general setup information about Elasticsearch is a good start. Especially of interest is the Install Elasticsearch on Windows (yes, we’ll do everything on Windows) and the important Elasticsearch settings.

We’ll find the same for Kibana: general setup information and the kibana configuration.

If you are interested in setup a production cluster, you should check Plan for production and also Resilience in small clusters. For Kibana you will be interested in that article: Use Kibana in a production environment.

That’s basically all we need for the start. In case you’re a bit overwhelmed by the amount of information, that’s fine. We’ll set up an Elastic Stack (single server), where I put all the important things together (but you’ll also find other quickstart guides).

Architecture

In our demo environment, we’ll have only one Elasticsearch and Kibana node. I use the latest available version 8.8.2, but the methodology should be the same. We also need a browser to access Kibana and to work with our Elasticsearch cluster.

Reference for the port communication Elasticsearch Networking and Kibana Port.

Setup Elasticsearch cluster

Elasticsearch

We’ll download the latest version and get the zip file. I create a new folder structure C:\Program Files\Elastic\Elasticsearch and copy the directory elasticsearch-8.8.2 into the Elasticsearch directory.

Theoretically, we could just start the Elasticsearch instance with a cmd.exe and run bin\elasticsearch.bat. But we want to configure a few things. How? We need to open the config\elasticsearch.yml.
The config file itself has some documentation and you can look up the full documentation in the reference article.

I show the finished config file beforehand, but we’ll dig into the certificates topic in the next chapter.

# Clustername and node name for a better naming
cluster.name: ElasticCitrixCluster
node.name: elk1
# Redirect data to another drive (otherwise it's in the C:\ drive which makes not that much sense)
path.data: X:\data
# Path to log files: 
path.logs: X:\logs
# Configure the address for the binding
network.host: 192.168.0.200
# Just to setup the cluster, we'll remove the setting after the cluster configuration.
cluster.initial_master_nodes: ["elk1"]

Start the Elasticsearch node

After the basic configuration, we can start our Elasticsearch instance (bin\elasticsearch.bat). After the start, you’ll see the following text in your cmd: Password for the elastic user.

That password is necessary, wait another 10-20 seconds then your Elasticsearch instance should be ready.

How can we check if the installation was successful? Open a browser and browse: https://elk1:9200 - because of the security feature you will get a prompt to enter credentials. The username is elastic and the password is the one from the cmd output.

If the installation was successful, you should see that:

[elk1] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN]

Congratulations, you installed Elasticsearch successfully.

Kibana

Now we install Kibana.
For that, we download the latest version and get the zip file. I create a new folder structure C:\Program Files\Elastic\Kibana and copy the directory kibana-8.8.2 into the Kibana directory.

Setup the kibana_system

Now is a good time to set up the passwords for our users. We’ll use elasticsearch-reset-password like that:

elasticsearch-reset-password -a -u kibana_system --url https://elk1:9200

Starting the Kibana instance

Our config file looks like that:

server.host: "elk1"
server.publicBaseUrl: "http://elk1:5601"
server.name: "elk1"

elasticsearch.hosts: [ "https://elk1:9200" ]

elasticsearch.username: "kibana_system"
elasticsearch.password: "kibanasystempassword"

# necessary because we use the built-in/auto tls security. Kibana will ignore that the certificate is self-signed.
elasticsearch.ssl.verificationMode: none

Now we can start the Kibana instance with kibana.bat. Now you can open a browser and open the website https://elk1:5601.
You will see a login mask. For now, we use the elk1 user with the generated password you have noted.
Congratulations, your Kibana instance is up and running.

Automatic start

Until now, we start Elasticsearch and Kibana manually. That is not very comfortable.
A solution is to implement Windows services, a easy way is the Non-Sucking Service Manager.

After downloading the nssm, we open an administrative cmd and use the following command:

nssm.exe install elasticsearch882

A dialog opens and we need to configure: path to the elasticsearch.bat. Under the category Details I configure the option Display name: to Elasticsearch 8.8.2. After that, we click on “Install service”

We do the same for Kibana (kibana.bat).

Check the healthiness of the Elasticsearch node

Congratulations, we achieved our goal. The final check is to open our Kibana instance https://elastic1:5601. After logging in, we open the menu and go to Management -> Stack Monitoring.
After enabling the automatic monitoring, we should see the following:

Everything is good, now we can start to install uberAgent.

Install uberAgent

First of all, we need to download uberAgent.

uberAgent itself provides a good starting point for the installation: Installing and Configuring Elasticsearch & Kibana.

We follow the instructions for the creation of the index template:

$cred = Get-Credential
# We use the elastic user

Invoke-RestMethod -Uri http://localhost:9200/_index_template/uberagent -Method Put -InFile .\elasticsearch-uberagent.json -ContentType "Application/json" -SkipCertificateCheck -Credential $cred

I added the parameter -SkipCertificateCheck because we use self-signed certificates. You need PowerShell 6 or higher.
The Elasticsearch cluster is secured with authorization and therefore we need the appropriate credentials.

When everything is fine, you will see this:

acknowledged
------------
        True

The barrier to using uberAgent in an ELK environment was that uberAgent itself did not provide dashboards. You would have to build these dashboards yourself, but fortunately, that is no longer necessary thanks to XOAP.

To import the dashboards we go to Management > Stack Management > Kibana > Saved Objects. Then in the upper right corner click on Import and import the file: Dashboard and Visualization.ndjson.

The installation of the uberAgent agent itself is described in the article Installing the Windows Endpoint Agent.

We run uberAgent_endpoint\bin\manual-install.cmd and we get a nice GUI installation.

In the Receiver Configuration step we choose HTTP Event Collector and add https://elk1:9200 to the Receiver(s): input field and elastic:elasticpassword to the App token input field.

The only thing we need to add manually in the config file (C:\Program Files\vast limits\uberAgent\uberAgent.conf) is to change Type = Splunk to Type = Elasticsearch.

If you have a license file, copy it to C:\Program Files\vast limits\uberAgent. Thanks to uberAgent who gave me a license file.

And because we use self-signed certificates, we need to add some configuration. uberAgent’s documentation helps us out here once again.

1. We will browse to https://elk1:9200 and download the CA certificate (Elasticsearch security auto-configuration HTTP CA). Then we will import the CA certificate in the computer context and in the Trusted Root Certification Authorities store. And yes had we done this earlier, we would have saved some “IgnoreTLSSelfSigned” settings on Elasticsearch / Kibana.
2. We’ll follow the recommendation and also set the ConfigFlags to TLSRevocationChecksBestEffort in the uberAgent.conf file.

Restart the uberAgent service and we’re ready to explore the uberAgent dashboards in our ELK stack!