XLSERVER USER
GUIDE
XLSERVER V1.0
1. Preface
OVERVIEW
Using this guide, you will learn how to:
●Understand and describe XLServer architecture
●The main goals and how to reduce your infrastructure costs
●Install and configure XLServer
●Use XLServer in order to provision the virtual databases
AUDIENCE
This document is intended for the Chief Information Officer (CIO), project managers and
database administrators interested in using XLServer for reducing their database
infrastructure costs.
2. Introduction to XLServer
XLServer is like a Database as a service(DBaaS). It is intended to help companies to
manage their database infrastructure efficiently.
The main goal of XLServer is to save infrastructure costs by reducing storage footprint,
software and hardware licensing costs and accelerate time to market of the projects.
The XLServer has functionalities that help you to accelerate your DevOps projects and do
faster deliveries. As an example, you can create a lot of production database clones in a
few seconds in order to test your application with fresh data. The clone databases use very
little disk storage, so you have no need to invest in a new storage array. Doing so, you will
save unnecessary storage waste, and mulutize lots of database instances in the same
server which helps you to save database software licences.
XLServer is a database visualisation platform with fast provisioning.
3. Architecture Overview
XLServer is a web application running in a self contained virtual machine. You have no need
for sharp expertise to install and configure the server.
You can run XLServer either in VMware or VirtualBox hypervisor.
XLServer is a J2EE web application exposed via REST API. End users can consume these
REST API via client over HTTPS or SSH. Inside XLServer there is a data store which is a
ZFS container to hold a copy of all datafiles of source databases. The datafiles copy called
golden image is continuously synchronised from source database in order to have a
consistent copy of source database.
Below picture describes how it works inside XLServer.
XLServer create initiale copy of the source database called golden copy. This copy is
updated continuously by doing synchronisation from the source database. Each time a
synchronisation happens, a new snapshot gets created to reflect the image of the source
database at this time. Using these snapshots, the end user can provision a new virtual
database by mounting the snapshot as a database container on the target server over NFS
protocol.
4. Download
XLServer is distributed as an OVA template file to be installed on oracle virtualbox or
vmware.
OVA can be downloaded from the XLServer home page at the following
URL:xls-server.com. Provide a valid account id and then choose to download virtualbox or
vmware OVA file. For the full version, a download link will be sent by email and it will be
valid only for 48 heures following subscription.
5. Installation
The installation of XLServer is the same under vmware or virtualbox. This section provides
a detailed procedure for installing OVA on your preferred virtual hypervisor. Assume that
you have downloaded the OVA under an accessible location on the server.
Import OVA file to EXSI or vmware workstation.
Open VMware Workstation Player.
Select Open a Virtual Machine.
Select the XLServer OVA file, and click Open.
Enter a name for the virtual machine, and select the directory where you want to
install the virtual machine and then click Import.
After the VM has imported, click Edit virtual machine settings.
●Memory: choose the amount of physical memory you want to use. It is
recommended to use at least 16GB of memory up to 64GB for very large
usage. For testing, you can keep the default value 2GB.
●Processors: choose the number of vCPU you want to use. It is recommended
to use at least 8 vCPU or more. For testing purposes, you can keep the
default value 1 vCPU.
●Network Adapter: choose host only adapter and click advanced to
regenerate MAC address. By default XLServer virtual machine is configured
with only one network adapter. You must use NIC supporting at least 10GB
for achieving high performance during virtual databases usage.
●Hard Disk (SCSI): By default there is only one hard disk attached to the
virtual machine for the operating system. In order to use XLServer, you must
add at least three disks. One disk or more for data storage. The size of disk to
use depends on the size and number of source databases to use. Two other
small disks for caching and log.
Click OK to save the configuration.
The installation of XLServer is done. The next step will heightligh how to configure
and connect to XLServer.
6. Configuration
XLServer offers command line interface for administration tasks and initial setup, graphical
user interface GUI for server configuration. Once you have connected to the server directly
or over ssh, you will have the possibility to interact with the command line interface for any
administration task. For each prompt, you can enter h(elp) to display available commands
or q(uit) to exit the prompt.
As a prerequisite
If you choose to use a static IP address for accessing XLServer, the IP address must be in
CIDR annotation like 192.168.60.30/24.
At this step, you need to reboot the virtual machine for changes to take effect. The
application service should be started automatically at reboot, if it is not the case, you do it
from the Command Line Interface.
NB: to choose the network interface to use, you have to select the UUID of NIC.
SERVICE MANAGEMENT
Login to server with sysman user and default password. At the prompt, enter “service”
then return. You can check if the service is running by using “status” command. To stop the
application service, enter “stop” then return. To start the application service, enter “start”
then return.
INITIAL CONFIGURATION
Once server configuration is done, you will be able to connect to XLServer via https with
URL https://<SERVER ADDRESS>:8443/
Login to server with ADMIN user and default password “admin”. At first connection, a
wizard helps you to configure the server step by step.
●Admin password: you have to change the default password for ADMIN user. The
default password is “admin”.
●Subscriber Details: enter the company information by which you have subscribed for
software download. The name and email address must be the same as your account
ID.
●Notifications: at this step, you have to enter SMTP server details. This screen is not
displayed for the trial version.
●Storage Configuration(DATA): this screen displays available luns discovered in the
server. You have to choose which luns you want to use for the data store.
●Storage Configuration(LOG): this screen displays available luns discovered in the
server. You have to choose which luns you want to use for the log.
●Storage Configuration(CACHE): this screen displays available luns discovered in the
server. You have to choose which luns you want to use for the cache.
●License Key: Enter software key you received after the download in the key field.
The confirmation code is a validation code you should receive by email during
configuration.
●Summary: at this screen, you need to review selected options, then click Finish to
finalize server configuration.
The server configuration takes a few minutes to finish. Once done, you have to reconnect
and enjoy.
7. Manage Environments
Environment management requiere to connect to a server with an ADMIN user or
privileged user. From the Environment page, you can add a new environment. Environment
can be either a single Linux host or Oracle cluster.
Once all fields are filled, click the SUBMIT button to validate environment creation.
For OS Credentials, you have the choice to use password authentication or public key. If
you decide to use a public key, this must be deployed first on the target server.
During the environment creation, all Oracle listeners, installed oracle homes and running
databases will be automatically discovered. Click on the newly created host to display
more details about the discovered components.
To display discovered databases, click on Databases link.
From this page, you can manually add the database if it is not discovered, or not up during
environment creation.
8. Manage Databases
The main core of XLServer is database management. We have three kinds of databases in
XLServer. Discovered database, connected Database or virtual database. Running
databases can be discovered during environment creation or refresh. At any time,
databases can be added manually. At this time, there is no container attached to the
database.
To be able to provision new virtual databases from the source database, you have to
connect the database by creating a container which will contain all database files as a
golden image copy. When you connect the discovered database, an image copy of this
database will be taken in a dedicated container. The golden image can be rolled forward at
regular intervals or on demande by doing synchronisation of the container with the source
database. A timeline will be created recording a snapshot of the container after each time
container synchronisation happens. So, the virtual database will reflect the image of the
source database on the time of the snapshot created.
CONNECT DATABASE
From the Database page, you can select the discovered database to connect. Click the
CONNECT button, connecting the database wizard will appear. You must have valid
database credentials with required privileges to connect the database.
On connection parameters, you must enter an existing and empty directory where the
container will be mounted in order to run database backup. By default, Oracle rman will
use two channels for the backup. You can choose any number of channels you want to use
depending on available resources on the server.
The last page of the wizard summarizes selected parameters that you need to check before
submitting.
Connecting database duration depends on the size of source database, number of channels
in use and network bandwidth. You can check the running job to track database connecting
progress. Once the database is connected, you can see more details on the database
details page.
PROVISION VIRTUAL DATABASE
From connected database detail, you can select a snapshot from which you want to create
a new virtual database.
On the snapshot detail, you have the creation date of the snapshot, the last applied SCN
and last change time on the Oracle source database. The virtual database will be created
as an image of the source database till this SCN. Click START PROVISIONING, and the
provisioning wizard will appear.
You have to choose the OS user, the target environment and oracle home installation
directory where you want to create the virtual database. Enter a valid database name in
field Database Name, and mount base where oracle datafiles will be mounted.
In the next page, you must enter a valid DB Unique Name and Instance Name for the
virtual database. By default, they are filled by the Database Name entered on the previous
page.
If you want to run a specific post SQL script on the virtual database after creation, enter the
full path name of your existing script.
You have the possibility to modify database creation parameters on the page Parameters.
By default, the virtual database parameters are inherited from the default database model
of the connected database if specified, otherwise, it will use parameters of the source
database.
The last page of the wizard, summarize all selected inputs for the virtual database creation.
Click FINISH to submit the job of provisioning. The virtual database creation will take a few
minutes.
Once the virtual database is created, it will be listed on the databases list, you can go to
detail and manage the virtual database.
9. Connected Databases
From the connected database page, you have the possibility to fully manage the source
database; synchronize the source container with incremental backup of the source
database, disconnect connected database, manual refresh, delete, specify a generic
postscript, the default database model, manage database credentials,...etc. All these
functionalities are deeply described in the next paragraphs.
MANAGE DATABASE SERVICES
You can add or delete database jdbc service from Source Configuration tab.
Click the add button to create a new jdbc service and fill all inputs.
If a service is not valid, the process of creation will fail.
If you want, you can remove a database service by clicking the delete button.
Be careful, if all database services are deleted, some connected database functionalities
which requiere source database access like synchronization will not work.
MANAGE DATABASE TEMPLATES
If there is no database template attached to the connected database, all subsequent virtual
databases will be created with source database parameters.
In order to customize a virtual database creation, you can create and attach a default
template to the connected database.
To attach a default database template to CDB, click on ATTACH TEMPLATE from DB
Template and choose a template to use.
To select another template, click on the button MODIFY TEMPLATE and choose another
template.
To detach any attached template, click on the button DETACH TEMPLATE.
DATA MASKING
A source database may contain sensitive data which is not visible to everyone. With
XLServer, you have the possibility to obfuscate sensitive data replicated into the virtual
database by replacing a real value by fictitious data during the provisioning.
XLServer provides some template to apply for masking sensitive data:
Random String: real value will be replaced by random string applicable only for string data type.
Random Number: real value will be replaced by random number applicable only for number data
type.
Random Date: real value will be replaced by random date applicable only for date data type.
Unset Value: real value will be replaced by null.
Sysdate: real value will be replaced by current date applicable only for date data type.
Constant String: real value will be replaced by a constant string applicable only for string data type.
Constant Number: real value will be replaced by a constant number applicable only for number data
type.
Constant Date: real value will be replaced by a constant date applicable only for date data type.
You can also use a custom template which needs to be created first.
To apply data masking, click on ADD TEMPLATE from the DATA MASKING tab. A data
masking wizard will appear.
The first step is to enter and validate database credentials in order to fetch tables
description to mask.
The next step is to choose a schema and select a table where data masking will be applied.
The page Attach Template will display all columns of the selected table. From this page,
you have the possibility to remove, modify or apply a data masking template for any
column.
Click the add button to select the masking template for the corresponding column.
You can repeat this step for all columns you want to mask.
On the next page, data masking sampling will be displayed in order to validate chosen
templates.
If it seems fine for you, you can click FINISH to add this template to the connected
database.
At each moment, you can modify or remove data masking templates from the Data
Masking tab.
The virtual database provisioning can take more time if a lot of columns and data are
concerned by the masking. By default, a virtual database at creation uses very less storage,
maybe a few MB, but can use more storage with data masking.
CONNECTION SETTINGS
By default, XLServer uses two channels to synchronize the connected database container
from the source database. If source database activity is very high and the underlying server
has more resources (CPU + RAM), you can increase the synchronisation parallelisme more
than two channels in order to speedup the synchronisation. This can be done by changing
the value of parallel connections input from Advanced Settings.
AUTOMATIC SYNCHRONIZATION
When you connect a source database, by default automatic synchronization is disabled. In
order to have the possibility to provision a new virtual database from different restore
points, you can manually synchronize the source container or enable automatic
synchronization in regular intervals.
To enable or modify scheduled time of synchronization, edit Synchronization Schedule on
Advanced Settings and select frequency you want.
There are three level of granularity of frequency:
Hourly: synchronization will run each hour at a fixed time.
Daily: synchronization will run once on day and time selected.
Weekly: synchronization will run once per week on day and time selected.
DATABASE CREDENTIALS
To modify database credential, edit DB Credentials from Summary tab, and enter database
username and password then validate. If credentials are not valid or the user has no
privileges on the source database, this process will fail.
GENERIC POSTSCRIPT
You can specify generic SQL postscript to run during the provisioning of a new virtual
database by editing Post Script Path and entering the path of existing SQL script. This
script must exist and be validated because failure of executing this script will stop the
process of provisioning.
ENABLE AND DISABLE
At each moment, you can disable or enable a connected database. Disabling the connected
database causes all writable actions on the source container to be disabled; no
synchronization, no provisioning of new virtual databases,...etc.
REFRESH
When you do major modifications on a source database like upgrade, you have to refresh
the connected database in order to reflect these changes on source containers. If it is not
done, maybe synchronization or provisioning new virtual databases will not work.
MANUAL SYNCHRONIZATION
If you want to take a snapshot of the source database in order to provision a new virtual
database from this snapshot, you can manually do the synchronization of the source
container. This can be done by clicking the Sync button on the header of the connected
database.
DELETE
To remove the connected database from XLServer, click on the delete button on the header
of the connected database. Doing this, the source container will be definitively removed
and you will not be able to create new virtual databases from this container.
The delete will fail if a source container has one or more virtual databases attached to it, or
if the connected database is in use (synchronization in progress as example).
When the connected database is deleted, you will still see the database name in XLServer
server as a discovered database.
10. Virtual Databases
A virtual database is a database mounted on a target environment and uses a snapshot of
the source database container at a specific point in time. At creation, the virtual database
uses very little disk storage, and its size may increase over time depending on the activity
generated on the database. Any data modification necessarily increases the disk size used,
unlike data viewing which has no impact on the size of the database.
It is highly recommended to reset or recreate the virtual database once its size becomes
very important.
ENABLE AND DISABLE
You can enable or disable the virtual database for maintenance raisons. When the virtual
database is disabled, all functionalities acting on the database are disabled.
START AND STOP
You can start or stop a virtual database directly from the database page. Stopping the
virtual database wil disable some functionalities like sharing or reseting.
RESET
Reset is a very powerful functionality of XLServer. You can reset a virtual database to its
initial state without passing from all steps of provisioning a new virtual database. It is
much faster than provisioning a new virtual database, because there is no need to recreate
a new container, no need for data masking or postscripts to be run. All data modifications
done on the virtual database will be lost after reset, and the size of the database will be
the same as it was created the first time.
SHARE
Sharing the virtual database is another powerful functionality of XLServer. You can share
your database with anyone by cloning the database. The clone database must be mounted
in a similar environment as the virtual database. The virtual database will be unavailable
during the cloning which can take very little time.
You can decide to share your database for troubleshooting as an example with your
colleague to take a look at the issue without needing to disturb you when using the same
database.
To share the database, click on the Branch Database button for virtual database page
detail. A wizard will appear.
Similar to the provisioning; you have to fill all inputs related to the target environment to
use.
Enter DB Unique Name and Instance Name for the new virtual database to create. Default
values are inherent from the database name selected on the previous page.
Review the selected values and click FINISH to supply the job.
The cloning of a virtual database is as fast as resting the virtual database. All postscripts,
data masking or database upgrade will be omitted.
DELETE
The virtual database delete is irreversible. The virtual database container will be destroyed
at the delete and all your data will be lost. You can delete your database by clicking the
Delete button from the virtual database detail page. If the virtual database is shared, you
must first delete all clones before deleting the virtual database.
MONITOR THE VIRTUAL DATABASE
Over time, the virtual database can suffer from performance due the size of the underlying
snapshot or the size of its clone database. You should monitor the size and usage of the
virtual database in order to avoid having performance penalties.
From the virtual database home page, you can check the size used by underlying snapshot,
the number of the snapshots of the virtual database which is linked to the number of
clones.
If the size of the snapshot is very big compared to the source database, you can delete the
clones if NB.Snapshot more than 1 which means that the virtual database has clones, or
reset the virtual database.
If you want to have fresh data, you can delete the virtual database and provision a new one
from the most recent snapshot of the source database.
11. Snapshots
A snapshot is a picture of the content of the database container at a specific point in time.
After connecting the discovered database, the initial snapshot gets created. Other
snapshots will be created each time a source database container gets synchronised either
manually or automatically. These snapshots serve to provision virtual databases to reflect
the source database at this point of time.
In the connected database detail page, a carousel of source database snapshots is
displayed. You can select and open any snapshot for more detail or provisioning.
From snapshot detail, you can find useful information about the snapshot.
In the Summary tab, the last applied SCN and time on the source database, the size and
creation time of the snapshot.
Data Container tab lists all datafiles of the source database and their size at the snapshot
creation.
Parameters tab lists the parameters of the database model if anyone is attached to the
connected database. If no model is attached to the connected database, the source
database parameters will be displayed. These parameters will be used to provision a new
virtual database.
From the snapshot detail page, you can delete a snapshot by clicking the Delete button or
provision a new virtual database by clicking the Start Provisioning button.
You will not be able to delete a snapshot if it is used by a virtual database. You must first,
delete all virtual databases created from this snapshot before deleting the snapshot itself.
12. XLServer administration console
You can fully manage XLServer from the System page. Monitor server health, resize disk
storage and many other settings.
SETTINGS
From the Settings section, you can add more storage disks if needed.
Click the RESIZE button to display the storage resize window.
Select disks you want to add to the data store. All disks must be in the same size as
existing disks on the data store.
Click OK to validate or CANCEL to discard the changes.
Always from this section, you can view subscription and network settings. You have a
possibility to update SMTP settings for notifications.
SERVER STATISTICS
From this area, you can monitor the server resources usage. You can view:
Disk storage usage
Datapool health
CPU usage
CPU load
Memory usage
Swap usage
Disk IO
USERS
You can manage application users from the USERS section.
You can :
view list of all users
Add a new user
You can choose the user to be either ADMINISTRATOR with all permissions or STANDARD
with specific permissions.
You can edit and modify the user settings
You can delete a user from the Delete button. The default user ADMIN cannot be deleted.
DATABASE TEMPLATES
During the provisioning, the virtual database will be created using parameters of the
database template if it is attached to the connected database. You can use a database
template to customize the virtual database settings.
You can manage these templates from the DATABASE TEMPLATES section. You can
create, modify or delete a template.
By default, there is only one template created named DEFAULT. You cannot modify the
values of default template parameters.
If you want to create a custom template, you can clone any existing database template and
then change the value of parameters.
One the template is cloned, you can edit it and change the value of parameters.
You can click a TRASH button to remove a template. The default template cannot be
removed.
DATA MASKING TEMPLATES
Data masking templates are patterns to use in order to mask sensitive data in the virtual
database.
XLServer provides some default templates that you can view from the DATA MASKING
TEMPLATES section.
You can create your own template by clicking on the CREATE CUSTOM DATAMASKING
TEMPLATE button.
You have to enter the template name, template description, domain and the template
expression.
In the template expression you can use any valid function. In order to avoid the failure of
provisioning when using a custom data masking template, you should validate the
expression directly in your database by SQL select statement.
The template expression can acte on string, number or date column type.
USER PREFERENCES
From this page, you can see your information as a connected user and change if need your
password.
LOGS
For troubleshooting, you can view the log of XLServer from the LOGS section. When you
are in this section, XLServer automatically fetches the latest log file from the server.
