2020
Access Control Extension preparation and various improvements
Processing Pipeline now supports iTraMS and other improvements
Access Control Extension and Raw Data Storage
December 21, 2020
Access Control Extension preparation
We are continuing our effort in preparing the Access Control Extension feature for Bosch IoT Insights. In this sprint, we migrated the internally used Bosch IoT Things subscriptions for our customers from Bosch IoT Cloud (BIC) service management to the Bosch IoT Suite portal (INS-1055), secured the functionality of Token Exchange with Bosch IoT Suite Auth (INS-1185) and changed the user roles to a dot-based name scheme, as the auth side changed its concept (INS-1348). Also, we reviewed what we have done so far (INS-1008) and planned the next steps (INS-1238). All this preparation is not visible to customers and will be deployed all at once in Q1 2021.
Data Quality topics
We strengthened the collaboration with the MongoDB developers. We now have the chance to test beta functionality with the existing databases. In addition, we established further methods and alerts (INS-971) to recognize and react faster to ui/data inconsistency problems.
Raw Data Storage preparation
We are still preparing a cheaper way of saving input data as raw storage and are working on different aspects to integrate this feature. As with the Access Control Extension, this feature is not visible to customers until we can deploy it.
Bug fixes
INS-1234: Invalid handling of 403 responses in the frontend, lead to annoying “you’re are logged out” behaviour
INS-1321: Processing logs call crashed project-management-service
INS-1371: Reprocessing job percentage by missing total count
INS-1329: Tour widget data cleansing did not work with playback widget
INS-1331: Booking a Free Plan failed when additional subscriptions besides Bosch IoT Things where active.
INS-1330: Text/data shown in the Illustration Widget did not align after saving
INS-1338: Quality Fix – CML Security findings
INS-1363: Fix of the Suite OAuth client identification
Update on the Access Control Extension
December 3, 2020
Access Control Extension preparation
We are continuing our effort in preparing the Access Control Extension feature for Bosch IoT Insights. In this sprint, we prepared parts of the existing user management (INS-1010) and now allow for lowercase roles (INS-1301). All this preparation is not visible to customers and will be deployed all at once at some point in the future.
Quality topics
To ensure the high-quality standards of Bosch IoT Insights, we cleaned out our codebase of outdated custom views. This should not have an impact on existing projects. Please contact our support team if you cannot access a specific view.
The project migration from custom processors towards the processing pipeline also continues; again a big customer project has been migrated successfully. In the same time frame, two projects with custom processors got booked.
Bug fixes
INS-1259: UI Action Button Widget did not encode some characters correctly
INS-1157: Wrong error message was displayed by data upload fail due to reached project quota
INS-1187: Device – Backdating UI Link event was not backdated by “create linked device”
INS-1189: Label was missing while using “create linked device” with backdating
INS-1235: Masterdata history: duplicate snapshot entries in device history
INS-1294: Http-Push: Authorization was not working properly on one endpoint
INS-1144: Project invitation mail was using the outdated projectlabel of projectconfig
Preparation for Access Control Extension continues
November 27, 2020
Access Control Extension preparation
We continue our effort in preparing the Access Control Extension feature for Bosch IoT Insights. In this sprint, we made sure that the existing api keys will still work (INS-1007) and changed roll conventions (INS-1266). All this preparation is not visible to customers and will be deployed all at once at some point in the future.
Bring your own Things supports OAuth2 Client (INS-1085)
The Bosch IoT Suite OAuth2 Client (Suite Auth Client) is now on the default authorization method for Bosch IoT Things. Therefore, we added this functionality to the Bosch IoT Insights configuration for a custom Bosch IoT Things connection. In the near future, we will perform a migration of all existing Bosch IoT Things connections.
Customers, who configured their own Bosch IoT Things instance for their Bosch IoT Insights project, will receive an E-Mail, asking them to create and enter their OAuth2 Client.
Device history available on other clouds (AWS & Azure)(INS-787)
The device history feature is now also available to Bosch IoT Things instances that are hosted on AWS and Azure. Before, this was only available for instances hosted on Bosch IoT Cloud. At the moment, achieving a working connection requires dedicated expertise and configuration effort with regard to a customer’s Bosch IoT Things solution. We plan to ease the configuration or totally automate the setup in the future. We will inform you when a more user-friendly approach is available. If you are interested to use this feature right now, please contact our support team.
Quality improvements
The test and code structure of the iTrams Decoder has been improved and aligned with other parsers (INS-1025). In addition, some third-party dependency weaknesses have been addressed (INS-1220 & INS-1275). These changes are not visible to customers.
The project migration from custom processors towards the processing pipeline also continues; preparations for later plan upgrades and premium customers have been finalized.
Bug fixes
INS-1123: Calendar action mail should now contain start and stop time
INS-1141: Car Media Lab Sending: Fixed downloaded zipped input document checksum error
INS-1188: Device UI: Missing backdating dialog by adding information block
INS-1199: Fixed width when editing Thing Attributes in Devices
INS-1259: Action Button Widget did not encode some characters correctly
INS-1189: Processing Pipeline UI: Processing Logs had problems rendering long error messages
Access Control Extension preparation and various improvements
November 9, 2020
Access Control Extension preparation
We took a big step in preparing the Access Control Extension feature. We achieved testing the credential migration mechanism (INS-1215), implementing the first two of three new authentication workflows (INS-1004 & INS-1006). In addition, we introduced a flag to prevent user management changes during the migration (INS-1009). All this preparation is not visible to customers and will be deployed all at once at some point in the future.
Quality topics
To ensure the high-quality standards of Bosch IoT Insights, we have fixed some security vulnerabilities. These are partially related to the spring migration we executed in the last sprint (INS-1159 & INS-1219). We also improved our existing workflow for monitoring new vulnerabilities (INS-1143) and are still preparing the migration of our OSS Scan Tool (INS-882).
Changed default role of project requester (INS-1150)
In the past, the person that booked a new project was automatically assigned the role “Project Manager”. We changed this to the role of “Project Admin”. This enables the project requester to execute delete operations. Please keep in mind that data deletion via the UI is only supported based on the input data. If you want to delete processed data, have a look at our API. Check out this overview of our current role concept.
The project migration from custom processors towards processing pipeline also continues; another big customer project has been migrated successfully.
Bug fixes
INS-943: MDF Parser Step/Function should only be available if the decoder functionality is activated for that project
INS-1100: Mdf Decoder: Time Vector issue
INS-928: Deletion of processed-data is now again possible with “compound” Id’s over the UI
INS-1079: UI Filter Widget: The widget did not react to query params change event
INS-1214: UI Filter Widget Selection Parameter Value and Label mixed up
INS-1183: UI-Map Widget: Fixed handling of missing GPS Positions
INS-948: Processing Pipeline UI: Selected Artifact was not shown in some cases
INS-1137: Processing Pipeline Input data will no longer be processed – fixed error in pipeline log
INS-1165: Processing Pipeline Increase required role for downloading pipeline code artifacts
INS-1130: Master Data Client: Fixed Resource leak caused by erroneous alive-checker
INS-1151: Master Data Client: Master Data History is not tracked any more for standard Things configuration
INS-1155: Data Upload: Tab Separated File Format caused Null Pointer Exception
INS-1192: Billig: Device Count data aggregation is not working properly
INS-1214: Calendar action mail contains missing start and stop time again
INS-1214: Device Management UI: width when editing Thing Attributes in Devices has been corrected
Technical maintenance and various improvements
October 26, 2020
Technical maintenance – spring migration (INS-580)
The main goal was to update outdated dependencies, which had the side effect that the UI & backend on average now answer 30% faster in terms of latency.
Pipeline Processing general availability (INS-1081 & INS-952)
We removed the “beta” flag from the pipeline processing feature of Bosch IoT Insights, as it now covers all major features from the custom processor we used before. Therefore, we can now announce the general availability of the feature. Also, the first heavy use customer migrated with a premium plan towards pipeline processing.
Several improvements for the devices tab (INS-1063 & INS-1064 & INS-1124 & INS-1125 & INS-1126)
When copying a device that has a device linked to it, it is now possible to also copy the device link. Regarding device representation, users can now insert line breaks in the device properties, which will then be displayed as expected. Furthermore, the feature name gets displayed in case that no feature definition is defined in the device type.
Devices as data source: When using a widget, it is now possible to filter for all devices without datatype and perform other queries on “all devices”. In addition, we implemented a new functionality to count devices with and without filters. See also the related Bosch IoT Things HTTP API.
Action Button widget (INS-823)
The newly introduced Action Button widget for paid plans allows you to execute http requests at the click of a button inside of Views. The https methods “get”, “put”, “post”, “delete” are supported as well as raw and form data and Insights documents. Please keep in mind that the remote address has to be whitelisted, which is currently only possible by support request.
Extension of the Data Table widget (INS-91)
The Data Table widget has been extended by the “Custom Column Definition”. Thus, users are able to choose what kind of data should be displayed and can define this either as a simple JSON property or a hyperlink.
A “styling” UI component addresses various widget styling use cases (INS-573)
We introduced a new color range styling element, which allows you to customize the representation more precisely – either by defining static coloring or using color ranges.
UI – jump from a widget data point directly to the corresponding data document or detail view (INS-735)
You can now enter hyperlinks into the scatter chart as tool tips. These can be used to jump to the corresponding data or data source.
Multiple tours in the Tour widget (INS-807)
The new option “Tour-End-Event” was added to tour map settings. This allows users to show multiple tours in one widget.
Option to number markers in the Location widget (INS-808)
We provided the location map widget settings with a new field called “Auto-Numerate Markers”. Thus, you can add numbers automatically to the markers displayed on the map and legend.
Improved zoom mode for charts (INS-820)
You can now define multiple new zoom modes in the Chart widget. They allow you to define which axis should be zoomable in the resulting chart.
Bug fixes
INS-632: Pipeline is using the wrong disk size amount at start
INS-909: Line Charts sometimes show two X-Axises instead of X and Y
INS-969: Query History does not start loading accordingly on first call
INS-1054: Service Broker DuplicateKeyException mongodb update
INS-1065: Extract Service Binding should use new CloudFoundry API
INS-1078: Longterm Calendar Booking is not displayed
INS-1086: MDF Decoder does not check for compatibility
INS-1093: Parser Step does only work as first step of pipeline
INS-1133: Error on Show Query button click
INS-1138: Gateway adds authorization header to permissions request also if not needed
INS-1146: iTrams – the timebucket selection does not use the selected value
INS-1152: Http data recorder now accepts files over 10MB
INS-966: Processing Information prevented browser crash
Improvements
INS-622: Additional option for Conditional Action to reduce logging size
INS-1202&INS-1036: First implementations in preparation of the migration towards suite auth
Small improvements and various bugfixes
October 6, 2020
Backdating device creation (INS-1021)
While device property changes could already be backdated, you now have the possibility to also backdate the device creation date itself. You can also apply a label and/or comment that will show up in your Device History tab.
The table widget is now pageable by defining a max count of rows (INS-1056)
Users that had a great amount of data in their table widget ran into the problem that their dashboards were getting too long. Now, there is an option for the table widget to define the number of table rows that should be displayed. The table can then be traversed by using pagination.
Prevention of invalid characters in Thing/Device properties (INS-903)
Naming a Thing property is now limited to those characters that are actually supported ([\s_a-zA-Z0-9-]). In the past users received an error message when using unsupported characters (like dots, slashes etc.).
Additional changes
We migrated all old Leaflet based map widgets to the newer MapBox Location Map widget. The deprecated Leaflet Location Map widget is no longer available.
Bug fixes
INS-272: Fixed an ODX-Decoder issue where string length definitions were not handled correctly
INS-925: Fixed a memory leak in the mdf4-Parser
INS-940: Configuring the Heatmap widget with out of bound latitude or longitude should not lead to an unresponsive behavior anymore
INS-942: Fixed a memory leak where the CML-Upstream could run out of disc space
INS-977: Fixed a problem where a Custom Processing Java step could not connect to Mongodb
INS-998: Fixed a problem where Reprocessing-Tasks could run forever
INS-1001: Fixed an DBC-Parser issue where negative offsets were not parsed correctly
INS-1026: Fixed a problem in the Processing Pipeline where Japanese characters led to a problem when processing data
INS-1047: Project admins should now be able to correctly create new collections in the Pipeline Processing Output Step
INS-1050: Fixed an issue where the Processing Pipeline could get stuck due to a step breakdown
INS-1066: Fixed an ODX-Decoder issue that some users were experiencing
INS-1072: Processing Pipeline Reprocessing should now correctly take the selected memory and disc quota into account
INS-1074: Fixed an issue where the Custom Processing steps that were written in Python would miss important ssl certificates
INS-1075: Fixed a Pipeline issue where log messages could get lost
INS-1077: Fixed a concurrency issue in the Processing Pipeline Parser Step
Processing Pipeline now supports conditional actions
September 17, 2020
The Processing Pipeline now performs the conditional action check during the output step
It is now possible to use the conditional action feature for processing pipelines. As before, users can configure the conditional actions under Services -> Conditional Action. The check if the data meets a condition is performed in the Output and iTrams Output step.
For all projects that are saving during the custom step, it is not possible to perform this check unless you send the data in addition to the output step.
To activate this feature, the pipeline version of your processing pipeline has to be higher than 1.53.0. You can check this in the app console by starting the processor (see picture below). If you see an older version number, please contact our support team.
Prevent Thing properties which contain invalid characters
Bosch IoT Things, which can be accessed in Bosch IoT Insights via the “Devices” tab, comes with character restrictions regarding the things and policy id. Therefore, the Insights UI now prevents the user from typing in these invalid characters.
Minor Updates
CML Upload app improved crash prevention
Translation in the Playback widget has been fixed
Various Pipeline Processing iTrams Parser improvements
Label-value order consistency improved in the UI
Migration of location widget from Leaflet to Mapbox
Versions 9 and following of the insights-step-maven plugin are now accessible
Upcoming UI style update
September 16, 2020
In the next days we will implement an update in our User Interface (UI) components. This will result in better readability and more clarity by adapting the current version of the Bosch Brand guidelines.
Among other things it will affect the style of fonts, input fields, forms, and tables for instance:
Processing Pipeline now supports iTraMS and other improvements
September 1, 2020
The Processing Pipeline provides an additional parser step for processing iTraMS based documents
The new step can be selected and configured in the Pipeline Configuration section:
Further information can be found in our online documentation.
Improved performance when loading processing information for the Processing Pipeline
To address insufficient performance when loading the Processing Logs entries for a certain Processing Pipeline, we implemented a corresponding performance optimization.
Improved start time of Processing Pipeline Apps
We improved the start time of the Processing Pipeline Apps using Docker containers. Moreover, the Docker container uses Debian Buster instead of Ubuntu as the underlying operating system. We expect no migration effort when existing apps will switch to the new Docker-based deployment, especially since the mentioned operating systems offer nearly the same feature set.
The migration process is automated when an existing app is stopped and started again.
The Pie Chart now also supports absolute beside relative values
Customers requested the ability to also show absolute values instead of relative ones. There is now an additional option for choosing absolute values in the Advanced Settings:
Leaflet replaced with MapBox
In the past, Bosch IoT Insights used Leaflet as a library for map visualization. Due to the fact that Leaflet is no longer maintained, we recreated the widgets using MapBox. All existing visualization using the old Leaflet version will be migrated to the new MapBox version during this week. The old Leaflet based widgets will be removed.
Database update to MongoDB 4.2
All our NoSQL replica sets were upgraded to MongoDB 4.2. All related changes can be found in the corresponding release notes.
In particular, we want to point out the new set of mathematical operators available for aggregation queries.
Small improvements and bug fixes
August 10, 2020
We have updated Bosch IoT Insights to provide various small improvements and bug fixes:
We fixed a problem in the Tour-Map widget (Mapbox) that caused the tour to disappear. This happened when users tried to switch between tabs by using the tab widget.
We fixed a problem in the Tour-Map widget (Mapbox) were the initial zoom level option was not taken into account. This happened when users tried to switch between tabs by using the tab widget.
Our new Device History should now correctly initialize, even if your device attributes contain a special character (dot “.”).
In some projects, an initialization failed because this character was used in the device properties.
If a Processing Pipeline instance crashed in the startup phase, there was no error shown in our console. This made it impossible for users to analyze the problem further. We resolved this issue now and any error even in the earliest phase will be returned to the user.
While using the Processing Pipeline Console an error could occur if the content contained JSON like text. Parsing and visualization of this content will now work for every case.
Using the new Pipeline Processing, it was not possible to reprocess single documents via Input History.
This has now been fixed.
If a user specified a default parameter in a query template, this could cause a conflict in the widget edit dialog – the user was not able to change the actual value anymore. Now, users can change the parameter accordingly.
SSL Certificate replacement pending
August 3, 2020
The SSL certificate for the domain https://bosch-iot-insights.com will be renewed since it expires on September 25, 2020.
If you hard coded the current certificate, you will need to update after renewal.
See our FAQs for further help: https://bosch-iot-insights.com/static-contents/docu/html/FAQ.html
New reprocessing feature for processing pipeline
July 28, 2020
Pipeline Processing: Reprocessing available
The UI has integrated the new reprocessing feature for processing pipeline now. Therefore, you can see the reprocessings of the last 24 hours under the tab “Pipeline (Beta)” below the list of pipelines. In addition, you get the status, the details and also the possibility to cancel a running reprocessing.
With the “Reprocess” button under the tab “Input History” you can start a reprocessing. You need to configure the pipeline, amount of instances and a query on the upcoming modal. It is easy to start a reprocessing beside the normal processing now.
Furthermore, the reprocessing is processed independent from the normal processing now. Reprocessing will not block the normal processing anymore.
Pipeline Processing: Improvements
Under the tab “Processing Logs” you can now see also log messages from on a custom step in the early start phase. Therefore it is easier to implement a custom step now.
The custom code examples are now more resilient against changes of the exchanged data format. As a result custom step which use this code will be more stable against changes from outside.
The Insights step plugin support Junit5 now. As a result, you have advanced testing possibility for your custom code.
Rollout of device history
With the Device History function you can track state changes of an Insights device. More precise, states of devices which are stored in the IoT Things service.
The following change events from Things are tracked by default:
CREATE and DELETE change actions.
Change actions which modify the deviceLinks feature / property of a device.
Custom properties of a feature.
You can also configure for each device type which changes should be tracked, easily. The log messages are stored in the processed data collection now. Therefore, you have the possibility to make queries on this data for your specific use cases.
Under the tab “Device Types” you can create a new device type. After that, you can configure which property changes should be tracked by the device history under “Information Block”.
For further details have a look into our documentation.
Under the tab “History” you can see the history log.
Query Template optional parameter behaviour with filter widget
The filter widget does not set optional parameter for an empty selection, to avoid unexpected behavior.
Location map group marker visible on all map layer types
On a map layer change the grouped makers remain visible.
New service update available for July
July 14, 2020
Pipeline Processing: Reprocessing API available
Project Admins can now use our new reprocessing feature for pipeline processing projects. At the moment, this is only possible via an API call under Bosch IoT Insights Swagger UI. The UI integration is still ongoing and will be available via upcoming updates.
Our new reprocessing feature follows a new approach, where the reprocessing and normal ongoing processing are now mostly decoupled. Only the database is still used by both. This leads to several benefits:
Reprocessing is no longer blocking the normal processing
Users can keep track of the reprocessing job status
It’s possible to stop a running reprocessing job
It’s possible to rescale the performance of a reprocessing job
Pipeline processing: DBC parser step available
Under the UI pipeline Configuration, Project Admins can now specify a vector BDC as a parser step for pipelines. You can now easily define and use Vector DBC definitions in our processing pipeline to decode any CAN-Trace.
See this example:
Decoder UI consolidation
The differentiation between decoder service and used decoders in the admin tab has been removed. Users can now specify everything regarding decoders in one place: Service -> Decoder.
Note: this tab is only visible for projects that are decoder-enabled (Standard plan or higher)
Query Template option to decide the database index
You can now specify the run-time setting of an index to design query templates. This advises the underlying database to use this index whenever possible, but cannot force it to. Thus, you can now influence the process of choosing an index and can make queries faster, especially when many almost similar indexes are defined.
Initial maximum zoom level for all MapBox widgets
Map widgets used in Bosch IoT Insights normally adjust the initial zoom level so that all points are visible.
In some cases, this meant that when only one point was given, the initial zoom level was way too high. To prevent this in the future, users can now define a maximum initial zoom level in every MapBox map widget. This functionality is not usable for leaflet maps, projects have to migrate to the newer MapBox maps if they want to use this functionality.
Clearer behavior of graph widgets when data is missing
The behavior of graph widgets regarding missing data has been reworked and is now more clear. This makes it easier for users to configure the widgets’ behavior according to their needs.
New service update available now
June 17, 2020
Upcoming pipeline reprocessing feature
The Python example insights_protocol.py was updated, because of a change in preparation of the upcoming pipeline reprocessing feature. This feature will allow you to keep better track of the documents that have been processed, including history and abort options. The newly added reprocessingId is the basis for that.
Please update your Python artifacts. Otherwise, they will break when you start a new version of your processing pipeline.
New Pipeline App Console shows all logs from the processing frame and stderr of custom code implementations
Under the new tab “App Console” you can now find all messages that your custom code running app produces. The logs, that were displayed under the console tab have been moved to the tab “Processing Logs”. This should help you to troubleshoot and develop custom artifacts.
Java custom code examples are now available for usage
The Java custom code examples are now available in the Bosch IoT Insights documentation. These examples have to be used in a different way compared to the Python custom code example. It is necessary to build the example with Maven before uploading. You can find a more detailed description in the README file in the example. Please keep in mind that pipeline processing is a beta feature.
New service update available
June 2, 2020
New Data Table Widget configuration options
You can now configure the columns of the data tables in a Bosch IoT Insights dashboard – making it easy to select which data fields you would like to see in your table.
An additional option is coloring the table cell based on its value. Either display a cell with a static background color or define the color according to configurable ranges.
Columns can also contain a link, that uses placeholders, which are then replaced with the data.
Unified decoder specifications management
In case your project is enabled to use decoder specifications, you will from now on find all your decoder specifications under Services → Decoder. The page Admin → Decoder has been removed.
We added new options when uploading a new decoder specification. If you are a project admin, you can now lock the decoder to be only removed by project admins. The expiration of the decoder specification is now optional.
Change of Bosch IoT Things automatic subscription behavior
When you book a new Bosch IoT Insights subscription, we automatically book a Bosch IoT Things free plan for you, which is connected to your Bosch IoT Insights project.
The Bosch IoT Things subscription is required to manage devices within Bosch IoT Insights.
In the past, we only created a new namespace within our Bosch IoT Things subscription.
In case you are using Bosch IoT Things more extensively, we recommend booking your own subscription and configuring the connection to Bosch IoT Insights with your new Bosch IoT Things subscription. For more information see: https://bosch-iot-insights.com/static-contents/docu/html/IoT-Things-Configuration.html
New service update available
February 6, 2020
General information:
The new SaaS pricing model is active since January 1.
Please see the Bosch IoT Suite service plans for further details (bookable via Bosch IoT Marketplace or AWS Marketplace).
Among others, the following features and improvements have been made:
Conditional Action activity log retention time
You can configure the retention time interactively via UI and API.
Please see the online documentation for further details.
See also the screenshot below:
Manual data path input for the illustration widget
It is not possible to manually define the data path using the illustration widget.
See also screenshot below:
Splitting of Query History
To increase latency performance the query history access log has been split to project space individual logs to increase the performance and reduce sporadic locks.
In addition, various minor bugs and improvements have been fixed/implemented:
INS-38
Mongodb query service returns $date in arrayINS-106
Replace all “v1” examples in the documentation with “v2”