This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Visualisation Assets

Manage assets within Visualisations

1: Visualisation Assets Example

Motivation

Visualisations are typically composed of multiple files or assets:

Javascript
CSS
HTML
Images

The Visualisation Assets system provides a way to manage these easily through the user interface.

User Interface

The Assets tab within a Visualisation allows you to upload and edit files within a folder structure. For example:

images/content/visualisation-asset-ui.png — Visualisation Assets Tab

In this example, all the assets are in the same directory. However, you can create a directory structure if necessary.

Operation

Save any changes and make them live
Save the live and any unsaved changes to a new document
Throw away any unsaved changes and revert to the live version
Add an asset to the tree. The asset could be a folder, a new empty file, or an uploaded file.
Delete an asset from the asset tree. Note that if you delete a folder, everything under the folder will also be deleted.
View the asset in the browser in a new window. This also allows you to see the URL or path of that asset, if you need the path for another asset.
Edit the name of an asset.
Note

Note that the extension of the file is significant as it is used for:
- The mimetype reported to the browser, and thus how the browser will present that file;
- The editor mode used for the file.
See Configuration below to find out how to customise these settings.

Editing Assets

Selecting (clicking on) an asset will display the asset in the editor, where possible.

Stroom tries to allow you to edit all assets within the browser. There are two restrictions:

The asset must not be larger than 512KiB. The asset’s content is transferred to the user-interface in RAM and thus the size must be restricted to avoid problems.
The asset’s content must be convertible to UTF-8 text.

Note that the file’s extension isn’t considered when attempting to load a file. As long as it is small enough and can be converted to UTF-8 text, it can be edited.

Note

Due to the potentially large size of assets, the user-interface implements a staged save where the asset is first saved to a staging area in the database. When you click the data is copied across to the live area of the database, where the asset can be downloaded. Generally this doesn’t make any difference to how you use the user-interface; however you will find that unsaved changes persist even though you’ve closed a tab or the browser.

To get back to the current live state and remove any unsaved changes, click the icon.

Import and Export, GitRepo Structure

The Import and Export or GitRepo format is designed to allow you to edit the assets within the import/export structure. If you look at the structure you’ll have something like this:

Name.Visualisation.UUID.json
Name.Visualisation.UUID.meta
Name.Visualisation.UUID.node
Name.Visualisation.UUID-path-assets/

Here Name is the name of the Visualisation document, and UUID is a long string that looks something like b565d110-508d-483c-95f7-69196479aee9.

The first three files (.json, .meta and .node) should not be edited. However, under the -path-assets/ directory you’ll find your assets in the same structure as shown in the Stroom user-interface. You may edit, add or delete files as required. When you re-import the files or Pull from the GitRepo, the files within Stroom will be updated.

Assets and Visualisations

The assets can be used directly via links from Scripts.

Alternatively you can define an asset named index.html in the root of your asset tree. If this exists then Stroom will load that as the visualisation.

HTTP Access

The assets are available via the URL /assets/<doc-id>/asset-path, where the <doc-id> is the ID of the document that owns the assets. The easy way to find the URL is by clicking the icon.

Note that this icon is only enabled when there are no unsaved changes.

In the example screenshot above, the file index.html has the URL https://localhost/assets/b565d110-508d-483c-95f7-69196479aee9/index.html.

Browser Cache

Note that your browser will cache the assets. This is a good thing as it will greatly speed up Stroom by reducing bandwidth requirements and server load.

Files will only be loaded when the Dashboard tab is opened. Changing file contents will only take effect if you close and reopen the tab. There is no need to refresh the browser or log out of Stroom.

The file /index.html will always be reloaded every time the tab is opened.

Other files will be reloaded if they have changed.

Thus the development procedure will look like this:

Make a change and save the Visualisation.
Open the Dashboard and click the Play button. You will see the result of your change.
Close the Dashboard
Go back to step 1

Links between Assets

If the assets are owned by the same document then you only need the relative path you defined within the tree. For example, theme-css.css or images/background.png.

If the assets are owned by different document then you’ll need the document ID as well. For example, ../b565d110-508d-483c-95f7-69196479aee9/common-css.css.

Since visualisations may be moved between servers, it is probably a good idea to avoid absolute paths and only use relative paths.

Configuration

The mappings from extension to mimetype and editor mode are configured in your local.yaml file.

appConfig:
  visualisationAsset:
    aceEditorModes:
        htm: "HTML"
        txt: "TEXT"
        css: "CSS"
        svg: "XML"
        xml: "XML"
        js: "JAVASCRIPT"
        html: "HTML"
    assetCacheDir: "asset_cache"
    clearAssetCacheOnStartup: false
    default: "application/octet-stream"
    defaultAceEditorMode: "TEXT"
    mimetypes:
        htm: "text/html"
        jpg: "image/jpeg"
        css: "text/css"
        tiff: "image/tiff"
        bmp: "image/bmp"
        apng: "image/apng"
        gif: "image/jpeg"
        svg: "image/svg+xml"
        png: "image/png"
        js: "text/javascript"
        webp: "image/webp"
        tif: "image/tiff"
        txt: "text/plain"
        xml: "application/xml"
        jpeg: "image/jpeg"
        html: "text/html"

aceEditorModes: You will normally want to leave these as the default values, but you may need to add a particular extension to suit your installation.
assetCacheDir: This is where the assets will be cached before serving them to the user interface. The master copy of the asset is kept in the database. There is a cache on each Stroom node to reduce database load.
clearAssetCacheOnStartup: If true then clear the asset cache each time Stroom starts up.

Note

Stroom does not clear assets from the cache by default. This could lead to a situation where the asset has been deleted within the UI, but is still available via HTTP/HTTPS. If this causes a problem you need to set this value to true and restart Stroom. The cache will then be recreated from valid content in the database.

default: The mimetype to send to the browser for an asset where the asset’s extension is not recognised. You probably don’t need to change this.
defaultAceEditorMode: The ACE editor mode to use when the asset’s extension is not recognised. You probably don’t need to change this.
mimetypes: The MIME type to send to the browser for each filename extension. You may need to add something to support a particular filename extension you use.

1 - Visualisation Assets Example

Walk through of converting a Script document visualisation into an Asset visualisation

Introduction

This document walks through the process of converting an existing Doughnut visualisation into an Asset-based visualisation.

Document the Source Visualisation

We’re converting System/Visualisations/Visualisations/Version3/Doughnut, so we’ll open it up and see how it is configured.

images/content/visualisation-document-doughnut.png — The original Visualisation Document we're going to convert

Function Name - We’re going to need to copy this to our new document.

Script - We’re going to leave this bit blank - our new document won’t reference a Script document.

Settings - We’re going to need to copy this to our new document.

Dependencies

Visualisation documents depend on a Script document. In turn, this depends on other Script documents.

The Doughnut Script document looks like this:

images/content/visualisation-script-document-script-doughnut.png — The original Script Document for the Doughnut visualisation

We’re going to need the script file, so copy-and-paste it into a text editor and save it somewhere.

We also need to look at the Settings tab which looks like this:

images/content/visualisation-script-document-settings-doughnut.png — The settings tab of the Doughnut Script document

This shows the Script documents that the Doughnut Script depends on. In turn, the dependencies can depend on other Script documents. We need to extract all these scripts so we can convert them into assets.

All Script documents hold Javascript. However, sometimes CSS is needed instead of Javascript, so the CSS is held in a Javascript String. This needs to be converted into plain Javascript to create a plain CSS file.

Convert CSS in Javascript into CSS

The Script will look something like this (cut down for this example):

(function() {
var cssStr = "" +
"/*" +
" * Copyright 2016 Crown Copyright" +
" *" +
" * Licensed under the Apache License, Version 2.0 (the \"License\");" +
" * you may not use this file except in compliance with the License." +
" * You may obtain a copy of the License at" +
" *" +
" *     http://www.apache.org/licenses/LICENSE-2.0" +
" *" +
" * Unless required by applicable law or agreed to in writing, software" +
" * distributed under the License is distributed on an \"AS IS\" BASIS," +
" * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied." +
" * See the License for the specific language governing permissions and" +
" * limitations under the License." +
" */" +
"@CHARSET \"UTF-8\";" +
"" +
"html, body {" +
"  width: 100%;" +
"  height: 100%;" +
"  padding: 0px;" +
"  margin: 0px;" +
"  overflow: hidden;" +
"  font-family: Roboto, arial, tahoma, verdana;" +
"  font-size: 13px;" +
"  font-weight: 400;" +
"}" +
"" +
"text {" +
"  fill: var(--vis-text-color);" +
"}" +
"" +
".vis {" +
"  margin: 0px;" +
"  padding: 0px;" +
"}" +
"" +
".vis-text {" +
"  shape-rendering: crispEdges;" +
"}" +
"" +
".vis-line {" +
"  stroke-width: 0.5px;" +
"}" +
"" +
".vis-area {" +
"}" +
"" +
".vis-axis {" +
"}" +
"" +
"::-webkit-scrollbar-corner {" +
"  background: transparent;" +
"}" +
"";
d3.select(document).select("head").insert("style").text(cssStr);
})();

You will need to remove the quotes, + signs, prefix and suffix Javascript to end up with this:

/*
 * Copyright 2016 Crown Copyright
 *
 * Licensed under the Apache License, Version 2.0 (the \"License\");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an \"AS IS\" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
@CHARSET \"UTF-8\";

html, body {
  width: 100%;
  height: 100%;
  padding: 0px;
  margin: 0px;
  overflow: hidden;
  font-family: Roboto, arial, tahoma, verdana;
  font-size: 13px;
  font-weight: 400;
}

text {
  fill: var(--vis-text-color);
}

.vis {
  margin: 0px;
  padding: 0px;
}

.vis-text {
  shape-rendering: crispEdges;
}

.vis-line {
  stroke-width: 0.5px;
}

.vis-area {
}

.vis-axis {
}

::-webkit-scrollbar-corner {
  background: transparent;
}

Regular expression search-and-replace is very useful for this, using the terms ^" and " \+$.

Resulting Assets

Once all this is done we end up with the following files stored in a folder on our local machine:

common-css.css
common.js
d3-grid.js
d3-tip.js
d3.js
doughnut.js
generic-grid.js
js-hashes.js
stroom-theme.css

The names are not important as long as they match up with the index.html file below.

Create a New Visualisation Document

Add the Assets

Select the Assets tab within your new Visualisation document.

Use the / Upload File menu items to upload all the CSS and Javascript files in turn. No folders are necessary - everything can go into the root directory.

Settings

Select the Settings tab within your new Visualisation document.

Function Name - Copy from the old visualisation, or just set it to visualisations.Doughnut.

Script - Leave this blank

Settings - Copy from the old visualisation. Make sure you get the whole file.

`index.html` Asset

We need an HTML file which will form the core of our visualisation. This must be named index.html to be considered for loading by Stroom.

It needs to look like this:

<html>
  <head>
    <meta charset="UTF-8">
    <link rel="stylesheet" href="theme-css.css" type="text/css" />
    <link rel="stylesheet" href="/ui/css/vis.css" type="text/css" />
    <link rel="stylesheet" href="common-css.css" type="text/css" />
    <script type="text/javascript" charset="UTF-8" src='../../../../../ui/vis.js'></script>
    <script type="text/javascript" charset="UTF-8" src='common.js'></script>
    <script type="text/javascript" charset="UTF-8" src='d3.js'></script>
    <script type="text/javascript" charset="UTF-8" src='d3-grid.js'></script>
    <script type="text/javascript" charset="UTF-8" src='d3-tip.js'></script>
    <script type="text/javascript" charset="UTF-8" src='generic-grid.js'></script>
    <script type="text/javascript" charset="UTF-8" src='js-hashes.js'></script>
    <script type="text/javascript" charset="UTF-8" src='doughnut.js'></script>
  </head>
  <body>
    <!-- Empty -->
  </body>
</html>

All the assets we’ve uploaded need to be linked into the HTML file.

Note that there are a couple of files that use the /ui/ path:

/ui/css/vis.css Stroom visualisation CSS
/ui/vis.js Holds the code that Stroom uses to communicate with the visualisation.

Both of these files must be included in your index.html for the visualisation to work.

Add to Dashboard

This new visualisation can be added to a Dashboard and configured in the usual way.

Visualisation Assets

Motivation

User Interface

Operation

Note

Editing Assets

Note

Import and Export, GitRepo Structure

Assets and Visualisations

HTTP Access

Browser Cache

Links between Assets

Configuration

Note

1 - Visualisation Assets Example

Introduction

Document the Source Visualisation

Dependencies

Convert CSS in Javascript into CSS

Resulting Assets

Create a New Visualisation Document

Add the Assets

Settings

index.html Asset

Add to Dashboard

`index.html` Asset