# RUAL Documentation: full documentation > Generated from https://docs.rual.nl, one markdown document with the key pages. Curated overview: https://docs.rual.nl/llms.txt. Block catalog: https://docs.rual.nl/llms-blocks.txt --- title: "Getting started with RUAL · RUAL Documentation" description: "Build web apps in weeks with RUAL: explore the blocks, blueprints, cluster APIs and guides in the official RUAL documentation." canonical: https://docs.rual.nl/ language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Getting started With RUAL, you can develop web applications in a matter of weeks, leaving the tedious tasks to us. Explore all the available options and resources in our comprehensive RUAL Docs. ## Quick Navigation - [New to RUAL? Start here](https://docs.rual.nl/getting-started/quickstart): Build your first blueprint in 15 minutes. A Hello World page and a working JSON API endpoint. - [First CRUD App tutorial](https://docs.rual.nl/getting-started/first-crud-app): Create, list, update, delete and search storage documents in a complete task-manager tutorial. - [Browse 1,900+ Blocks](https://docs.rual.nl/block-types): Explore our extensive library of over 1,900 pre-built blocks for every need: arrays, conditions, APIs, integrations, and more. - [Cluster APIs](https://docs.rual.nl/cluster/api): Access 52 REST APIs for managing blueprints, users, storage, and system operations. - [User Management](https://docs.rual.nl/cluster/user-access-management): Learn how to manage user access, permissions, and security for your RUAL clusters. - [Release Notes](https://docs.rual.nl/core-versions): Stay updated with the latest features, improvements, and bug fixes across all RUAL versions. - [Blueprint Guide](https://docs.rual.nl/blueprints/introduction): Master the art of visual programming with RUAL's powerful blueprinting system. - [Tips & Tricks](https://docs.rual.nl/blueprints/tips-and-tricks): Discover advanced techniques for working with large blueprints and complex workflows. ## Popular Block Categories - [`array` Blocks](https://docs.rual.nl/block-types/array): 99 blocks for array manipulation, filtering, and transformations. - [`date` Blocks](https://docs.rual.nl/block-types/date): 104 blocks for date operations, formatting, and calculations. - [`condition` Blocks](https://docs.rual.nl/block-types/condition): 68 blocks for logical operations and flow control. - [`files` Blocks](https://docs.rual.nl/block-types/files): 63 blocks for file operations, uploads, and processing. - [`discord` Blocks](https://docs.rual.nl/block-types/discord): 4 blocks for Discord bot development and integration. - [`http` Blocks](https://docs.rual.nl/block-types/http%20connection): 41 blocks for HTTP requests and API integrations. ## Getting Started with RUAL The most frequently utilized aspect during development in [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio) is what we call [blueprinting](https://docs.rual.nl/blueprints/introduction). When mentioning RUAL Studio, we are referring to the cluster management pages accessible at [rual.at (opens in new tab)](https://rual.at). Within the [blueprinting](https://docs.rual.nl/blueprints/introduction) section, certain features are designed to simplify your work with blueprints. We recommend reading the [tips & tricks](https://docs.rual.nl/blueprints/tips-and-tricks) in the blueprints menu. This resource offers valuable insights on navigating large blueprints, as well as tips on copying & pasting and creating groups around multiple blocks. ## Cluster Access & Authentication To use any cluster through [RUAL Studio (opens in new tab)](https://rual.at), you must have a cluster URL and a valid username and password that are authorized for RUAL Studio Access. For more information on managing user access, please refer to our [User Access Management (UCM) resource](https://docs.rual.nl/cluster/user-access-management). If you are not already an administrator, an existing administrator will need to create an account for you and grant the appropriate access levels. This process ensures that only authorized users can access specific clusters, maintaining security and control within the RUAL Studio environment. Not sure which core version to build on? Production clusters should run the latest stable. See the [Version Management guide](https://docs.rual.nl/architecture/version-guide) for channels, safe upgrade routines, and where breaking changes are documented. ## API Integration Each cluster offers both public and private APIs, which can streamline the authorization process for developing custom frontends, as well as hybrid or native applications. For those building hybrid native apps using [React Native (opens in new tab)](https://reactnative.dev/) and looking to connect with our WebSocket, we offer [helper code](https://docs.rual.nl/other/react-native). The most up-to-date documentation for Cluster APIs can be found in the left-side menu under "Cluster APIs." Alternatively, you can [click here](https://docs.rual.nl/cluster/api) for a comprehensive overview of the available APIs. ## Recent Core Releases - [v15.0.0](https://docs.rual.nl/core-versions/150000): Latest development release - [v14.0.6](https://docs.rual.nl/core-versions/140006): Current stable release - [v14.0.5](https://docs.rual.nl/core-versions/140005): Previous Release - [v14.0.4](https://docs.rual.nl/core-versions/140004): Previous Release - [v14.0.3](https://docs.rual.nl/core-versions/140003): Previous Release - [v14.0.2](https://docs.rual.nl/core-versions/140002): Previous Release ## Frequently asked **What is RUAL?** RUAL is a platform where you build web applications from blueprints: visual flows of connected blocks that run on a cluster. Pages, APIs, storage and scheduled work are all built on the same canvas, without writing backend code. **Do I need to write code to build an application in RUAL?** No. Pages, API endpoints, storage queries and background jobs are built by connecting blocks on a blueprint canvas. You can add your own React components when you want custom front-end behaviour, but that is optional. **Where do I start with RUAL?** Start with the Quickstart, which takes about 15 minutes and ends with a live Hello World page and a working JSON API endpoint. After that, the First CRUD App tutorial covers creating, listing, updating, deleting and searching documents. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Quickstart · RUAL Documentation" description: "Build your first RUAL blueprint in 15 minutes: create a Hello World page, activate it, then expose a JSON API endpoint and call it with curl." canonical: https://docs.rual.nl/getting-started/quickstart language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Quickstart: Your First Blueprint in 15 Minutes Go from cluster login to a live Hello World page and a JSON API endpoint: created, activated, and tested with blueprints, without writing backend code. In this quickstart you build two things on your cluster: a **page** that renders "Hello World" in the browser, and an **API endpoint** that returns it as JSON. Along the way you practice the four moves every RUAL developer uses daily: creating a blueprint, placing and connecting blocks, activating, and testing in development view. ## 1. Prerequisites Before you start, you need: - **A cluster URL**. The address of the RUAL cluster you will build on. - **A username and password** authorized for RUAL Studio access. If you don't have an account yet, an existing administrator creates one for you. See [Getting Access](https://docs.rual.nl/cluster/getting-access). You work in [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio), the web application at [rual.at](https://rual.at). Log in with your cluster URL and credentials to open your cluster. ## 2. Create Your First Blueprint All logic in RUAL lives in blueprints: large canvases where you place blocks and connect them into flows. Open the blueprints overview in RUAL Studio and create a new blueprint. RUAL first asks *what type of blueprint you would like*; the chosen type pre-arranges the canvas with the right starting blocks. - Choose `Build a UI Page`. This seeds the canvas with a [`state_page`](https://docs.rual.nl/block-types/state%20ui/state_page) block, the starting block of every page flow. - Give the blueprint a clear name, for example `Hello World Page`. One blueprint per component is the convention, similar to assigning a single class to a file in traditional coding. - The canvas opens. Set the URL on the `state_page` block, for example `/hello-world`. ## 3. Build the Page A page flow needs a render block and at least one element to display. You will use four blocks: | Block | Display Name | Role | | --- | --- | --- | | [`state_page`](https://docs.rual.nl/block-types/state%20ui/state_page) | new page | Starts the flow when someone opens `/hello-world`. | | [`state_render_page`](https://docs.rual.nl/block-types/state%20ui/state_render_page) | render | Sends the combined HTML to the frontend and renders the page. | | [`state_h1`](https://docs.rual.nl/block-types/state%20ui/state_h1) | h1 | The heading element that displays your text. | | [`value_default`](https://docs.rual.nl/block-types/value/value_default) | value | Holds a fixed text value, here `Hello World`. | Now wire them together: - **Add the render block.** Left-click the `flow` out-pin of `state_page`, drag into an empty area of the canvas, and pick `render` from the search results. The connection is established automatically. Also drag the `Connection` out-pin onto the render block's `connection` in-pin. - **Set title and description.** The render block requires a `title` and a `description`. `Right-click` the canvas, choose `Add new block`, add a `value_default` block for each, and connect them. - **Add the heading.** Add a `state_h1` block the same way and connect its `state` out-pin to the render block's `content` in-pin. - **Set the text.** Add one more `value_default` block, type `Hello World` into it, and connect it to the h1 block's `text` in-pin. - **Save.** Use the `Save` button in the bottom bar, or `Cmd/Ctrl + S`. You just used the two connection kinds every blueprint is built from. `flow` pins control *when* a block runs, the execution order. Data pins such as `value`, `object`, and `state` carry *what* a block works on. You can only connect pins of compatible types. Read more in [Block Execution](https://docs.rual.nl/blueprints/block-execution). ## 4. Activate and Test Pages, APIs, and modals are inactive by default. Click `Activate` in the top bar of the blueprint to make your page respond. Then test the page in your browser: - Open the page from RUAL Studio. Studio automatically redirects you with the `?development` query, so with a RUAL Developer account you see your saved, not yet deployed, blueprint. - You should see a plain page with your **Hello World** heading. If not, check that you saved and activated the blueprint. - To execute the non-deployed blueprint against production data, select `Production Run` within your blueprint. Visitors only see the page after you deploy it to production. Saving keeps changes in development; deploying is a separate step. See [How to Deploy](https://docs.rual.nl/deployment/how-to-deploy). ## 5. Your First API Endpoint An API endpoint is a blueprint too. Create a new blueprint of type `Build an API endpoint` and name it, for example, `Hello World API`. The canvas is seeded with an [`on_startup_register_uri_get`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_get) block, which exposes a GET URI under `/api/`. Set the URI to `hello-world`: your endpoint becomes `/api/hello-world`. Build the response flow with these blocks: | Block | Display Name | Role | | --- | --- | --- | | [`on_startup_register_uri_get`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_get) | on api get | Registers `GET /api/hello-world` and starts the flow on each request. | | [`trigger_custom_function`](https://docs.rual.nl/block-types/globals%2Cfunction%20execution/trigger_custom_function) | create | The function that runs on every request. | | [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) | reply in json | Sends the JSON body and status code back to the caller. | | [`httpconnection_current_request`](https://docs.rual.nl/block-types/http%20connection/httpconnection_current_request) | current request | Reference to the incoming HTTP connection. | | [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields) | new object | Builds the `{"message": ...}` object. | | [`value_default`](https://docs.rual.nl/block-types/value/value_default) | value | The `Hello World` text. | | [`number_default`](https://docs.rual.nl/block-types/number/number_default) | number | The `200` status code. | - **Create the handler function.** Drag from the `On Request` out-pin into an empty area and create a function. Name it, for example, `hello_world`. Its `flow` out-pin runs on every request. - **Add the reply block.** Add a [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) block and connect the function's `flow` out-pin to its `flow` in-pin. - **Connect the request.** Add a [`httpconnection_current_request`](https://docs.rual.nl/block-types/http%20connection/httpconnection_current_request) block and connect it to the `connection` in-pin. - **Set the status code.** Add a `number_default` block with `200` and connect it to the `code` in-pin. - **Build the JSON body.** Add an [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields) block with a field `message`, connect a `value_default` holding `Hello World` to that field, and connect the object out-pin to the `data` in-pin: objects are cast to a JSON value automatically. - **Save and activate.** `Cmd/Ctrl + S`, then `Activate` in the top bar. Call your endpoint from any terminal: ``` curl https:///api/hello-world ``` ``` { "message": "Hello World" } ``` New pages and APIs use the `*public` scope by default, so anyone can call them. To restrict access, click the lock icon on the block to open the Scopes Management Modal and assign scopes. See [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control). Your cluster also ships 40+ built-in REST APIs for managing blueprints, users, and storage, documented under [Cluster APIs](https://docs.rual.nl/cluster/api). ## 6. Next Steps You created two blueprints, wired flow and data pins, activated a page and an API, and tested both. Where to go from here: - [Core Concepts](https://docs.rual.nl/getting-started/core-concepts): Blueprints, blocks, pins, flows, storage, and deployment. The mental model behind everything you just did. - [Build Your First CRUD App](https://docs.rual.nl/getting-started/first-crud-app): A full tutorial: create, list, update, delete, and search tasks with blueprints and built-in storage. - [Tips & Tricks](https://docs.rual.nl/blueprints/tips-and-tricks): Work faster on the canvas: drag-to-connect, auto-created blocks, selection, search, and the minimap. ## Frequently asked **How do I create my first blueprint in RUAL?** Open the blueprints overview in RUAL Studio and create a new blueprint, then pick the blueprint type. Choosing Build a UI Page seeds the canvas with a state_page block, which is the starting block of every page flow. **Why does my RUAL page or API not respond after I save it?** Saving is not enough. A blueprint only starts responding after you press Activate in the top bar. Until then the page or endpoint stays silent, even though the blueprint is saved. **How do I test a RUAL endpoint before deploying it?** Append ?development to the URL. That calls the saved version of the blueprint. Without it you are calling the last deployed version, which does not include your unsaved or undeployed changes. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Build Your First CRUD App · RUAL Documentation" description: "Step-by-step RUAL tutorial: plan a schema-less task document, then create, list, update, delete, and search tasks with blueprint blocks, no code needed." canonical: https://docs.rual.nl/getting-started/first-crud-app language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Build Your First CRUD App A step-by-step tutorial: plan a schema-less task document, then create, list, update, delete, and search tasks with blueprints, no code required. In this tutorial you build a small **Tasks** manager: a page that lists tasks, a form to create them, a detail view, and edit, delete, and search functionality. Everything runs on blueprints and the built-in storage. You will not write any backend code. Before you start, make sure you have access to a cluster (see [Getting Access](https://docs.rual.nl/cluster/getting-access)) and that you know your way around the blueprint canvas (see [Blueprint Introduction](https://docs.rual.nl/blueprints/introduction)). Remember to hit `Activate` in the top bar of each blueprint, or your pages will not respond. ## 1. Plan Your Data RUAL storage is JSON-based and schema-less. You don't create tables or declare fields up front. The tradeoff is that there are no relationships between documents, so each document carries everything you want to display (denormalization). A task document looks like this: ``` { "owner_guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "owner_name": "joe doe", "status": "open", "title": "write the release notes" } ``` Three things to notice: - **Everything is lowercase.** Field names are always stored in lowercase, and searches are case-sensitive: storing searchable values in lowercase saves you from missed matches later. - **The owner is denormalized.** Instead of only referencing a user, we embed `owner_name` so the list page can show it without a second lookup. We keep `owner_guid` as the reference for when we do need the full user document. - **There is no `created` field.** Creation and update timestamps live in `_meta` (`_meta.created`, `_meta.updated`). Never in the document root. Do not set your own copy of them. Every document automatically gets a `_meta` object with its unique `guid`, timestamps, and removal state. You will see it in the output of the create step below. For the full storage reference. See [Storage](https://docs.rual.nl/blueprints/storage). ## 2. Create a Task Create a new blueprint of type `Build a UI Page`. This seeds the canvas with a [`state_page`](https://docs.rual.nl/block-types/state%20ui/state_page) block. Build a small form with a [`state_form`](https://docs.rual.nl/block-types/state%20ui/state_form) containing a [`state_input_dynamic`](https://docs.rual.nl/block-types/state%20ui/state_input_dynamic) (field `title`) and a [`state_button`](https://docs.rual.nl/block-types/state%20ui/state_button). When the button is clicked, the create flow runs: - Get the submitted values with [`state_form_get`](https://docs.rual.nl/block-types/state%20ui/state_form_get), which outputs the form data as an object. - Add a [`storage`](https://docs.rual.nl/block-types/globals%2Cstorage/storage) block and select your `tasks` storage. - Add a [`mutations_set_bp_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_set_bp_field_multiple) block and select the fields to set: `title` from the form, `status` = `open`, and the owner fields from the current user. You never set a creation timestamp yourself: storage writes `_meta.created` for you. - Add a [`function_create_document`](https://docs.rual.nl/block-types/storage/function_create_document_from_mutations) block. Connect the storage and mutations to its in-pins, and connect the flow pin from the button's click event. The block returns the created document: including the generated `_meta` with its `guid`: on the `object` out-pin, and the raw guid on the `guid` out-pin: ``` { "_meta": { "cms": 1752768000362, "created": 1752768000, "entity": 1, "expiry": -1, "guid": "b34686f50e1b04964250f13391274d85308c129a793c6ec97bc41870fc67081e", "removed": 0, "ums": 1752768000803, "update_hash": "60e27209fa93063b5605e41605c2722ed428cae0", "updated": 1752768000 }, "owner_guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "owner_name": "joe doe", "status": "open", "title": "write the release notes" } ``` ## 3. Build the List Page The list page is another UI page blueprint. Its flow searches the `tasks` storage and hands the results to a table. Build the search query first: queries are composed by connecting query blocks and feeding them into [`function_search`](https://docs.rual.nl/block-types/storage/function_search): | Block | Purpose | Connects To | | --- | --- | --- | | `storage` (tasks) | Selects the storage to search in | `function_search` (storage pin) | | [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) (status = open) | Match only open tasks | [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) | | [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) | Wrap the term query in an efficient filter context | `query_and` | | [`query_sort_field`](https://docs.rual.nl/block-types/query/query_sort_field) (_meta.created, desc) | Newest tasks first | `query_and` | | `query_and` | Combine filter and sort into one query | `function_search` (query pin) | | `number_default` (25) | Limit the result set | `function_search` (limit pin) | `function_search` outputs an `array` of matching documents. Render them with an [`state_advanced_datatable`](https://docs.rual.nl/block-types/state%20ui/state_advanced_datatable): select the `tasks` storage on the block, pick the columns (`title`, `status`, `owner_name`), and place its `state` out-pin on the page. To render custom cells. A status badge, an edit button per row. Use the table's row component out-pin; see [Iterations](https://docs.rual.nl/interfaces/iterations) for how iterative content works. ## 4. Show Task Details For the detail view, create a UI page with the task guid in its URL. The `state_page` block exposes the URL parameters on its `params` out-pin: take the `guid` from there and feed it into a [`function_get_document`](https://docs.rual.nl/block-types/storage/function_get_document) block with the `tasks` storage selected. Getting by guid is the fastest retrieval method and easier to cache than searching. The block continues on the `found` flow pin when the document exists, so render the fields there and show a "task not found" state on the other path. To load several tasks at once: for example all tasks of a project. Use [`function_get_documents`](https://docs.rual.nl/block-types/storage/function_get_documents) with an array of guids. ## 5. Update and Delete Updates go through mutations, RUAL's transaction-based update system: each update is processed sequentially, so two users editing the same task can't overwrite each other. To mark a task as done, use [`function_update_document`](https://docs.rual.nl/block-types/storage/function_update_document_mutations) with the task's `guid` and a [`mutations_set_bp_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_set_bp_field_multiple) that sets `status` to `done`. The same mutation blocks you used at creation work here: increments, array adds, field removals; see [Updating Documents](https://docs.rual.nl/blueprints/storage#updating-documents) for the full list. For deletion you have two options: - [`function_remove_document`](https://docs.rual.nl/block-types/storage/function_remove_document), **soft delete**. The document is marked as removed and excluded from search results, but the data stays and can be brought back with [`function_restore_document`](https://docs.rual.nl/block-types/storage/function_restore_document). - [`function_delete_document`](https://docs.rual.nl/block-types/storage/function_delete_document), **permanent delete**. The document is gone for good (unless revisions are enabled on the storage). > **Best Practice** Use `remove` (soft delete) in most cases. It allows recovery when a user deletes a task by accident, and you can permanently purge removed documents later with a [repeating event](https://docs.rual.nl/blueprints/repeating-events) after a retention period. ## 6. Add Search Finally, let users find tasks by title. Add a [`state_input_dynamic`](https://docs.rual.nl/block-types/state%20ui/state_input_dynamic) (field `q`) above the list and trigger the search flow on enter or on a button click. The flow builds a full-text query and runs the same `function_search` as before: | Block | Purpose | Connects To | | --- | --- | --- | | [`query_bool_simple_query_string_field`](https://docs.rual.nl/block-types/query/query_bool_simple_query_string_field) (field: title) | Full-text match of the user's input on `title` | `query_and` | | [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) (status = open) | Keep the open-tasks filter from step 3 | `query_and` | | [`query_sort_field`](https://docs.rual.nl/block-types/query/query_sort_field) (_meta.created, desc) | Keep the sorting | `query_and` | | `query_and` | Combine text query, filter, and sort | `function_search` (query pin) | > **Case-Sensitive Searches** Storage searches are **case-sensitive**. Since we stored titles in lowercase, convert the user's input to lowercase before connecting it to the query block, or `Report` will not match `report`. [`query_bool_simple_query_string_field`](https://docs.rual.nl/block-types/query/query_bool_simple_query_string_field) handles common user search syntax, which makes it the right choice for user-facing inputs. For autocomplete-style or typo-tolerant search. See the other text query blocks in [Full-Text Search](https://docs.rual.nl/blueprints/storage#full-text-search). ## 7. Next Steps You now have a working CRUD app: documents created with mutations, listed with a composed query, read by guid, updated transactionally, and soft-deleted. From here: - [Storage](https://docs.rual.nl/blueprints/storage): Go deeper on query types, denormalization strategies, document lifecycle, and caching. - [Storage Events](https://docs.rual.nl/blueprints/storage-events): React to creates, updates, and removals: for example to sync denormalized owner names. - [Cluster APIs](https://docs.rual.nl/cluster/api): Expose your tasks to external services through 40+ REST APIs. ## Frequently asked **Where does RUAL store the creation date of a document?** In the _meta object, as _meta.created, together with _meta.updated, _meta.guid and the removal state. Storage writes those fields for you, so you never add your own created field at the root of the document. **Does RUAL storage support relationships between documents?** No. Storage is JSON-based and schema-less, and there are no relationships between documents. Each document carries the values it needs to be displayed, which is why examples embed a name next to the referenced guid. **Why are my RUAL storage searches not finding anything?** Field names are always stored in lowercase and searches are case-sensitive. Store searchable values in lowercase when you write them, so the query matches later. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Core Concepts · RUAL Documentation" description: "The core concepts behind RUAL blueprints, blocks, storage and execution." canonical: https://docs.rual.nl/getting-started/core-concepts language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Core Concepts The key ideas behind RUAL. Blueprints, blocks, flows, storage, and deployment. ## What is RUAL? RUAL is a no-code platform for building ERP/CRM systems, apps, and websites. You build all backend logic by placing blocks on a visual canvas, there is no custom backend code. The frontend can be built with blocks too, or with [custom React components](https://docs.rual.nl/interfaces/components) for more advanced use cases. You manage your system in [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio), the web application at [rual.at](https://rual.at). From there you manage blueprints, traffic, queue, deployment, translations, templates, and base settings for your cluster. ## Blueprints A blueprint is a large canvas where you design workflows. You place blocks on it and connect them to create flows. Each blueprint is typically dedicated to a specific component: a page, modal, API, or a set of functions, similar to assigning a single class to a file in traditional coding. Multiple users can work on the same blueprint at the same time. Team members can add comments, group blocks with titles, and copy blocks between blueprints. RUAL logs all blueprint modifications automatically, like git commits but with real-time collaboration. Since [version 13.0.7](https://docs.rual.nl/core-versions/130007), blueprints have separate development and production modes. Saving does not deploy to production, you must explicitly deploy through the [deployment](https://docs.rual.nl/deployment/how-to-deploy) menu. - [Blueprint Introduction](https://docs.rual.nl/blueprints/introduction): Learn more about navigating blueprints, context menus, and saving changes. ## Blocks Blocks are the building units of every flow. There are over 1000 available, from array and date operations to HTTP requests and Discord integrations. Each block has input pins (in-pins) and output pins (out-pins) that define what data goes in and what comes out. #### Pin Types Pins are types, you can only connect pins of compatible types. The most common ones: - `flow`: controls the execution order between blocks, determining *when* a block runs. - `value`. A string value, such as a name, email, or identifier. - `object`. A JSON object with key-value pairs. - [Browse All Blocks](https://docs.rual.nl/block-types): Explore the full library of over 1000 blocks organized by group. ## Flows A flow is a group of connected blocks within a blueprint. Each flow has a starting block that determines its type and purpose. A single blueprint can contain multiple flows of different types. #### Flow Types The following flow types exist in RUAL, each starting with a specific block: | Flow Type | Starting Block | Purpose | | --- | --- | --- | | **Page** | [`state_page`](https://docs.rual.nl/block-types/state%20ui/state_page) | Builds a user-facing page with UI components. Pages are accessible via URLs and can be public or restricted with [scopes](https://docs.rual.nl/blueprints/remote-access-control). | | **Function** | [`trigger_custom_function`](https://docs.rual.nl/block-types/globals%2Cfunction%20execution/trigger_custom_function) | Creates a reusable function that can be called from other flows or blueprints. Functions are private by default. | | **API** | `on_startup_register_uri_{method}` | Registers an API endpoint (GET, POST, PUT, PATCH, DELETE, or file upload) that can be called by external services or your frontend. | | **Modal** | [`state_flow_modal`](https://docs.rual.nl/block-types/state%20ui/state_flow_modal) | Creates a pop-up modal that can be shown on a page. | | **Storage Event** | [`storage_event`](https://docs.rual.nl/block-types/storage/storage_event) | Executes when a document is created, updated, or removed in a specific storage. | | **Repeating Event** | [`schedule_repeating_event`](https://docs.rual.nl/block-types/events/schedule_repeating_event) | Executes at a recurring interval, similar to `setInterval()` or `crontab`. | ## How Flows Work Together In a typical app, a Page flow renders a form where users input data. When the user submits, a Function validates the input and creates a document in storage. A Storage Event picks up that write and sends a confirmation email in the background. An API endpoint lets external services query the same data, and a Repeating Event runs every hour to generate statistics and cache them in Redis. Functions can be called from any flow type using the [`function_custom_execute`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute) block. Public functions can be called across blueprints, while private functions are only accessible within their own blueprint. ## Storage RUAL includes a built-in, enterprise-scale storage system with full-text search. Storage is JSON-based and schema-less. You don't need to predefine structures. It does not support relationships between entities, so include all necessary data within each document (denormalization). Documents are updated using mutations, a transaction-based system that processes updates sequentially. This makes counters and array operations safe to use without race conditions. - [Storage](https://docs.rual.nl/blueprints/storage): Learn how to create documents, build search queries, model data, and manage document lifecycles. ## Activation and Deployment Before any pages, APIs, modals, or events in a blueprint become active, you need to activate the blueprint using the `Activate` button in the top bar. All elements are inactive by default. Since [version 13.0.7](https://docs.rual.nl/core-versions/130007), there is a clear split between development and production. Saving keeps changes in development mode. To make them available to users, you need to [deploy](https://docs.rual.nl/deployment/how-to-deploy) the blueprint to production. > **Getting Started Tip** If you are new to RUAL, start by reading the [Blueprint Introduction](https://docs.rual.nl/blueprints/introduction) to learn how to navigate and work with blueprints, then explore the [Tips & Tricks](https://docs.rual.nl/blueprints/tips-and-tricks) for productivity shortcuts. ## Next Steps - [Block Execution](https://docs.rual.nl/blueprints/block-execution): Learn how blocks execute, including parallel flows, dataflow, and error handling. - [Browse Blocks](https://docs.rual.nl/block-types): Explore the full library of over 1000 pre-built blocks. - [Cluster APIs](https://docs.rual.nl/cluster/api): Access 40+ REST APIs for managing your cluster. ## Frequently asked **What is the difference between a blueprint and a block in RUAL?** A blueprint is the canvas that holds one piece of logic, such as a page, an API endpoint or a scheduled job. A block is a single step inside it. You build a flow by connecting blocks on the blueprint. **What is the difference between activating and deploying in RUAL?** Activating makes the saved version of a blueprint respond in the development view. Deploying publishes it to production. They are separate steps, so you can test a change before your users see it. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Support · RUAL Documentation" description: "Get help with RUAL: the checks that solve most problems, what to include in a report, and where to go when you are still stuck." canonical: https://docs.rual.nl/support language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Support Something not working, or documentation that does not match what you see in RUAL Studio? Start here. Most issues are solved by the checks below, and the rest reach us faster when you bring the right details. ## The documentation is wrong or unclear Use the **Tell us what to improve** link at the bottom of any page. It opens a short form that sends the page you were reading along with your message, so we can fix the exact paragraph you were looking at. No account or ticket needed. ## Before you report a platform problem These four checks resolve the majority of reports, and they are also the first four questions we would ask you: - **Is the blueprint activated?** A saved blueprint does not respond until you hit `Activate` in the top bar. Endpoints and pages stay silent otherwise. - **Are you testing the deployed or the saved version?** Append `?development` to call the saved version. Without it you are hitting the last deployment. - **What does the console say?** The blueprint console shows the executed path, the error pin, and the values that flowed into it. See [Debugging Blueprints](https://docs.rual.nl/troubleshooting/debugging). - **Did it work before?** Compare against block revisions and the audit log to find the change that broke it. See [Common Issues & Fixes](https://docs.rual.nl/troubleshooting/common-issues). > **Check the release notes first** A behaviour change after an upgrade is often documented. The [core versions](https://docs.rual.nl/core-versions) pages list what changed per release, and [Version Management](https://docs.rual.nl/architecture/version-guide) explains the channels and safe upgrade routines. ## What to include in a report A report with these details is usually answered in one round instead of three: | Detail | Why we need it | | --- | --- | | Cluster URL | Identifies the environment and its core version. | | Core version | Shown in the sidebar card and on the [core versions](https://docs.rual.nl/core-versions) pages. | | Blueprint name and `guid` | Lets us find the exact flow instead of guessing from a screenshot. | | Timestamp of a failing run | Makes the run findable in the cluster logs and the API history. | | Steps to reproduce | What you clicked or called, what you expected, what happened instead. | | Error text or error pin output | The literal message, not a paraphrase. | Never send passwords, access tokens or refresh tokens. If a token is involved, send its `guid` from the token list instead of the token itself. ## Where to go next - [Common Issues & Fixes](https://docs.rual.nl/troubleshooting/common-issues): The failures we see most: blueprints that do not respond, scope errors, cache surprises, and how to fix each one. - [Debugging Blueprints](https://docs.rual.nl/troubleshooting/debugging): Read the console, use the simulation popup, inspect error pins and block revisions. - [Error Code Reference](https://docs.rual.nl/reference/error-code-reference): What each error code means and what usually causes it. - [Getting Access](https://docs.rual.nl/cluster/getting-access): No cluster URL, username or password yet? An existing administrator creates your account. ## Still stuck If the checks above did not resolve it, contact the RUAL team through [rual.nl/support](https://rual.nl/support) with the details from the table above. Access requests for a cluster you do not have an account on go through an administrator inside your own organisation, not through us. ## Frequently asked **How do I report a problem with RUAL?** Run the four checks first: is the blueprint activated, are you testing the development or the deployed version, what does the console say, and did it work before. If it still fails, report it with the cluster URL, core version, blueprint guid, a timestamp of a failing run and the exact error text. **How do I report an error in the RUAL documentation?** Use the Tell us what to improve link at the bottom of any documentation page. It opens a short form that sends your message together with the page you were reading. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Cluster · RUAL Documentation" description: "Everything around the cluster itself: getting an account, managing who may do what, and exposing your own REST endpoints." canonical: https://docs.rual.nl/cluster language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Cluster Everything around the cluster itself: getting an account, managing who may do what, and exposing your own REST endpoints. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Getting Access to a Cluster · RUAL Documentation" description: "Learn how to get access to a RUAL cluster: request an account from an administrator, log in for the first time, understand cluster URLs, and get an API token." canonical: https://docs.rual.nl/cluster/getting-access language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Getting Access to a Cluster Everything you need before you can build: an account, your first login, the right URLs, and an API token. Every RUAL cluster is its own secured environment. Before you can open [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio) or call a cluster API, an administrator of that cluster needs to create an account for you. This guide walks you through getting credentials, signing in for the first time, and setting up your development environment. ## 1. Obtain Cluster Credentials RUAL clusters do not have public sign-up. Accounts are created by an existing administrator through [User Access Management (UCM)](https://docs.rual.nl/cluster/user-access-management), who also decides which permissions your account gets. - Contact an administrator of the cluster you want to work on and request an account. If you are new to the organization and don't know who manages the cluster, ask the person or team that invited you to the project. - The administrator creates your user in the `All users` section of RUAL Studio and assigns the appropriate permissions for your role. - You receive three things: the **cluster URL**, your **username**, and a **password**. Keep these private. They identify you in the cluster's audit log. ## 2. First-Time Login - Open your cluster URL in a browser. You are greeted by the cluster login screen. - Enter your username and password and sign in. - Depending on how your cluster is configured, you may be asked to change your password or set up two-factor authentication on first use. Follow the on-screen instructions. - After signing in you land in RUAL Studio, where the available menu items reflect the permissions on your account. > **Locked Out by a Custom Login Page?** Clusters can replace the default login page with a custom one built in blueprints. If that page breaks and you can no longer reach the login form, append `?studio=request` to the URL to regain access to the default login page. See [Overwrite Default Pages](https://docs.rual.nl/interfaces/overwrite-default-pages) for details. ## 3. Understanding Cluster URLs You will work with two kinds of URLs: the RUAL Studio application, and your own cluster URL that serves the pages, apps, and APIs you build. | URL | What it serves | | --- | --- | | [rual.at](https://rual.at) | RUAL Studio. The cluster management application where you build blueprints and manage your cluster. | | ` ` | The production view of your cluster: the deployed pages and APIs your end users see. | | ` ?development` | The development view. When logged in with a RUAL Developer User account, you see the non-deployed version of pages; RUAL Studio redirects you to this view automatically. | | ` ?studio=request` | Forces the default login page, used to recover from a broken custom login page. | Saving work in a blueprint never affects production. Changes only reach the production URL after an explicit [deploy](https://docs.rual.nl/deployment/how-to-deploy). ## 4. Setting Up Your Development Environment RUAL runs in the cloud, so there is nothing to install. You need three things: - A modern web browser. - Your cluster URL. - Your username and password. There are two distinct ways to work with a cluster, and it helps to separate them early: | Access type | Used for | Authenticated by | | --- | --- | --- | | **RUAL Studio access** | Building blueprints, managing users, storage, and deployment through the browser. | Username and password authorized for RUAL Studio Access. | | **API access** | Calling cluster APIs from custom frontends, scripts, or external services. | An `access_token` obtained through the login APIs. | To get an API token, a user logs in through the login APIs of the cluster, which return an `access_token`. Tokens are valid for 14 days and are extended automatically while actively used. You can pass a token as an `Authorization: Bearer` header, as `x-authtoken` or `x-token`, as a cookie named `access_token`, or in the URL as `?access_token=`. See [Cluster APIs](https://docs.rual.nl/cluster/api) for the full authentication reference. ## 5. Roles & Permissions What you can see and do in a cluster is controlled by scopes. Administrators add scope values to the `custom_scopes` array on your user document, or assign you to [user groups](https://docs.rual.nl/cluster/user-access-management#user-groups) that bundle scopes for a role such as Sales or Support. Additional account settings: for example `is_root_user` for cluster administrators, control security-sensitive behavior. The full permission list is documented in [User Access Management](https://docs.rual.nl/cluster/user-access-management). Security best practices for administrators and users alike: - **Least privilege**: grant only the scopes a user needs for their role, preferably through user groups instead of individual scopes. - **Protect credentials**: treat passwords and access tokens as secrets; never commit them to code or share them between users. - **Use the audit log**: sensitive actions such as removing blueprints and accessing secured system settings are logged: review the log regularly. ## Next Steps - [Quickstart](https://docs.rual.nl/getting-started/quickstart): Build your first working flow on the cluster in minutes. - [Core Concepts](https://docs.rual.nl/getting-started/core-concepts): Learn the key ideas behind RUAL: blueprints, blocks, flows, storage, and deployment. - [User Access Management](https://docs.rual.nl/cluster/user-access-management): Full permission list, user groups, and account security settings. ## Frequently asked **How do I get access to a RUAL cluster?** RUAL clusters have no public sign-up. An existing administrator creates your account in User Access Management and assigns the permissions for your role. You then receive three things: the cluster URL, your username and a password. **I lost my RUAL cluster URL, where do I find it?** The cluster URL is issued together with your account, so ask an administrator of that cluster. Every cluster is its own secured environment, and the URL identifies which one you are signing in to. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Cluster - User Access Management · RUAL Documentation" description: "Within your cluster, you have the capability to create various types of users, each specifically tailored for roles such as developers, customers, or visitors." canonical: https://docs.rual.nl/cluster/user-access-management language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # User Access Management (UCM) Create and manage cluster users in RUAL Studio: accounts, permissions, user groups, security settings, and auditing. Within your cluster, you have the capability to create various types of users, each specifically tailored for roles such as developers, customers, or visitors. This guide covers the full account lifecycle: creating users, assigning permissions, organizing user groups, securing accounts, and auditing access. It is written for cluster administrators. If you need an account yourself, start with [Getting Access](https://docs.rual.nl/cluster/getting-access). For a deeper look at how roles and permissions fit together. See [User Roles Explained](https://docs.rual.nl/cluster/user-roles-explained). ## Users Overview In the [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio), when you access the `All users` section from the left-side menu, you will find a comprehensive list of all active users within your cluster. Users marked with a `*` (star) symbol possess full access to all resources within the cluster, subject to any specific settings enabled or disabled on their accounts. ## Creating a User RUAL clusters have no public sign-up. Every account is created by an administrator. Creating and editing users requires the `Manage User Permissions` permission or full (`*`) access. - Open [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio), select your cluster, and go to `All users` in the cluster settings section of the left-side menu. - Click `Create user` to open the create dialog. - Fill in the user details: a **username** and a **password**, plus any profile fields your cluster uses. Share the credentials with the user privately: [Getting Access](https://docs.rual.nl/cluster/getting-access) describes what a new user needs for their first login. - Save the user. The account is created immediately and appears in the `All users` list. - Open the new user from the list to fine-tune the account: enable [permissions](#permissions), set [account settings](#settings), add [custom scopes](#giving-user-permission), or assign [user groups](#user-groups). Every user is stored as a document in the cluster, so anything you can set in the dialog can also be managed from blueprints or through the [cluster APIs](https://docs.rual.nl/cluster/api). ## Permissions This section details each permission available in the RUAL Studio and its specific function. - `Cluster Admin`: This permission is typically granted to a select few who require complete access to all resources within the cluster. - `Blueprints: All Access`: Grants the ability to view and remove any blueprint. This permission is exclusively available to developers. - `Wildcard Scope`: Provides comprehensive access to any Cluster, custom API, and custom page created in RUAL. This is essential for developers to create new resources. - `Staging Developer`: Enables a developer to view, modify, and simulate blueprints. - `Production Developer`: Allows a developer to view, modify, and deploy blueprints to production. - `Blueprints: Production Run`: Controls the `Run: Production` feature in play mode within blueprints. - `Blueprints: Activate Blueprints`: Grants access to activate or deactivate blueprints. - `Blueprints: Remove Blueprints`: Enables users to remove blueprints, with all actions recorded in the audit log. - `Blueprints: Deploy Blueprints`: Enables users to deploy blueprints to production. - `Manage User Permissions`: Required to manage other users' permissions. Users without this cannot modify accounts with higher permissions. - `Manage System Settings`: Allows users to modify, view, and create system settings. Access to secured system settings is logged. - `Manage Storages`: Permits users to create, update, or delete storages within blueprints. - `Manage Domains`: Enables users to create, update, or delete domains within the cluster. - `Manage Custom Components`: Enables users to create, update, or delete React Components within the cluster. - `View Cluster Statistics:`: Enables users to view statistics for this cluster on the cluster statistics page or request them with API. #### Permission matrix The same permissions grouped by area, with what they allow and the user type each is typically granted to: | Area | Permission | What it allows | Typical user type | | --- | --- | --- | --- | | Blueprint editing | `Wildcard Scope` | Access to any cluster, custom API, and custom page created in RUAL; required to create new resources | Every developer | | `Staging Developer` | View, modify, and simulate blueprints | Developer | | | `Blueprints: All Access` | View and remove any blueprint | Lead developer | | | `Manage Storages` | Create, update, or delete storages within blueprints | Developer | | | `Manage Custom Components` | Create, update, or delete React Components | Frontend developer | | | Production | `Production Developer` | View, modify, and deploy blueprints to production | Senior developer / release manager | | `Blueprints: Production Run` | Use `Run: Production` in play mode | Developer debugging against live data | | | `Blueprints: Activate Blueprints` | Activate or deactivate blueprints | Developer | | | `Blueprints: Deploy Blueprints` | Deploy blueprints to production | Release manager | | | `Blueprints: Remove Blueprints` | Remove blueprints (recorded in the audit log) | Lead developer | | | User management | `Manage User Permissions` | Create and edit users and manage their permissions; cannot modify accounts with higher permissions | Administrator | | Security & cluster | `Cluster Admin` | Complete access to all resources within the cluster | Cluster owner | | `Manage System Settings` | Modify, view, and create system settings; access to secured settings is logged | Administrator | | | `Manage Domains` | Create, update, or delete domains within the cluster | Administrator | | | `View Cluster Statistics` | View the cluster statistics page or request statistics through the API | Operations / viewer | | ## Example Access Levels Most accounts fall into one of a few typical access levels. Use these as a starting point and adjust to your team: | Access level | Typical setup | What the user can do | | --- | --- | --- | | **Full-access administrator** | Starred (`*`) account with `Cluster Admin` and the `is_root_user` setting enabled | Everything: blueprints, deployment, user management, and system settings. Reserve for one or two cluster owners. | | **Production developer** | `Wildcard Scope`, `Staging Developer`, `Production Developer`, `Blueprints: Activate Blueprints`, `Blueprints: Deploy Blueprints`: without `Manage User Permissions` | Edit and save blueprints and deploy them to production, but cannot manage users or security settings. | | **Read-only viewer** | No developer or manage permissions; only view scopes through `custom_scopes` (optionally `View Cluster Statistics`) | Sign in and view the pages, APIs, and statistics their scopes allow, without modifying anything. | | **API-only integration user** | A regular account with only the `custom_scopes` of the endpoints it needs to call | Authenticates through the login APIs and calls cluster APIs with an `access_token`. Never intended for RUAL Studio work. See [Getting Access](https://docs.rual.nl/cluster/getting-access#development-environment) and [Cluster APIs](https://docs.rual.nl/cluster/api). | > **Start with the minimum** It is easier to grant a missing permission later than to clean up after an over-privileged account. Prefer [user groups](#user-groups) over individual scopes so access stays reviewable. ## Giving user a permission To assign a permission to a user, add the permission name as a value in the `custom_scopes` array field within the user's document. This enables the user to access the associated pages or APIs as defined by the scope. Example of a user granted two scopes: ``` { "custom_scopes": [ "view_customers", "create_orders" ], "username": "john.doe" } ``` ## User Groups User groups provide a convenient way to manage permissions for multiple users by organizing them into logical groups. Instead of assigning scopes to each user individually, you can create groups with predefined scopes and assign users to these groups. #### Creating User Groups User groups are stored in the `usergroups` database. Each group contains: - `name`: A descriptive name for the group (e.g., "Sales", "Finance", "Support") - `scope`: An array of scope values that define all permissions for users in this group Example of a user group document stored in the `usergroups` database: ``` { "name": "Sales", "scope": [ "view_customers", "create_orders", "view_reports" ] } ``` #### Assigning Users to Groups To assign a user to one or more groups, add the group GUIDs to the `groups_guid` array field in the user's document. When a user is assigned to multiple groups, they receive all unique scopes from all groups they belong to. #### Example Use Cases | Group Name | Example Scopes | Use Case | | --- | --- | --- | | `Sales` | `view_customers`, `create_orders`, `view_reports` | Sales team members who need access to customer data and order creation | | `Finance` | `view_reports`, `manage_invoices`, `view_payments` | Finance team members who need access to financial data and reports | | `Support` | `view_customers`, `view_tickets`, `create_notes` | Support team members who need to help customers and manage tickets | Example of a user assigned to both Sales and Finance groups: ``` { "custom_scopes": [], "groups_guid": [ "94ad734891564818888bea7e2519aedd", "1c5bfaf8fb034f8498d204c5072b554e" ], "username": "john.doe" } ``` In this example, the user will receive all unique scopes from both the Sales and Finance groups, plus any individual scopes defined in their own `custom_scopes` field. ## Account Settings Settings in RUAL provide fine-grained control and customization options for various user accounts. While some settings align with permissions discussed earlier, there are some distinct ones worth noting. | Key | Description | | --- | --- | | `allow_login_as_user` | When this setting is enabled `on`, it grants the ability to utilize the [user signin as](https://docs.rual.nl/block-types/users/function_user_signin_as) function for user accounts. If this setting is not enabled for an account, you won't be able to employ the [user signin as](https://docs.rual.nl/block-types/users/function_user_signin_as) function within blueprints for that account. Please note that this setting cannot be enabled for accounts with the `is_root_user` setting enabled. | | `is_root_user` | This setting is enabled for user accounts designated as cluster administrators. Such accounts enjoy additional security measures against login attacks, password forget requests, and sign-in functionality. | | `setting_manage_users` | User accounts with this setting set to `on` have the authority to configure the `allow_login_as_user` setting for other accounts. If the target account already exists, only users with this setting enabled can configure it. | Example object of settings data ``` { "settings": [ { "key": "allow_login_as_user", "value": "off" }, { "key": "is_root_user", "value": "on" } ] } ``` ## Editing, Resetting, and Removing Users Click any user in the `All users` list to open their detail page, where you can change permissions, settings, custom scopes, and group membership, then save. Keep in mind that users without `Manage User Permissions` cannot modify accounts with higher permissions than their own. Each row in the users table also has quick actions: - `Activity`: opens the [User Activity](#auditing) page for that account, so you can review what the user has been doing in the cluster. - `Reset`: resets the failed login attempt counter. RUAL blocks repeated unsuccessful authentication attempts, so use this to unlock a user who was locked out after too many wrong passwords. - `Delete`: permanently removes the account and revokes the user's access to the cluster. Use this when someone leaves the project. It cannot be undone. ## Requesting Access If you need access to a cluster, you cannot create an account yourself. An administrator of that cluster must create one for you. - **You know an administrator:** send them your name and the role you need. [Getting Access](https://docs.rual.nl/cluster/getting-access) walks you through receiving credentials and your first login. - **You don't know who manages the cluster:** ask the person or team that invited you to the project. They can point you to the cluster owner. - **You are the administrator and locked out by a broken custom login page:** append `?studio=request` to the cluster URL to reach the default login page. See [Overwrite Default Pages](https://docs.rual.nl/interfaces/overwrite-default-pages). ## Security Best Practices - **Least privilege**: grant only the permissions a user needs for their role, preferably through [user groups](#user-groups) instead of individual scopes. - **Limit full-access accounts**: keep the number of starred (`*`) users small, and enable `is_root_user` on them for additional protection against login attacks, password forget requests, and sign-in abuse. - **Be careful with `allow_login_as_user`**: it lets blueprints sign in as that account through the [user signin as](https://docs.rual.nl/block-types/users/function_user_signin_as) function. Enable it only for accounts that genuinely need impersonation (for example support flows), and control who may enable it with `setting_manage_users`. It can never be enabled on `is_root_user` accounts. - **Review access regularly**: delete accounts of people who left the project, and unlock legitimate users with the `Reset` action instead of sharing accounts. ## Auditing User Access The **User Activity** page shows what an individual account has been doing in the cluster. Open it through the `Activity` action on a user in the `All users` table. Activity data is also available through the cluster APIs with the `user_activity_view` scope, for example to feed an external monitoring tool. Sensitive cluster actions are recorded in the audit log as well. Every blueprint modification and removal, and every access to a secured system setting. Review the audit log after incidents and on a regular schedule; see [Audit log navigation](https://docs.rual.nl/troubleshooting/debugging#audit-log) for where to find it. ## Next Steps - [Getting Access](https://docs.rual.nl/cluster/getting-access): The new-user perspective: obtaining credentials, first login, cluster URLs, and API tokens. - [User Roles Explained](https://docs.rual.nl/cluster/user-roles-explained): How roles, permissions, and scopes fit together across a cluster. - [Access Token Device Information](https://docs.rual.nl/cluster/user-token-info): Device and browser details available on every access token. - [Cluster APIs](https://docs.rual.nl/cluster/api): Authenticate API-only users and call cluster endpoints with access tokens. ## Frequently asked **How do I create a user in RUAL?** In RUAL Studio, open the Users section under cluster settings and use Create User. You set the account details and then assign the permissions or user group that fits the person's role. **How do I reset a user's access in RUAL?** The users table has a Reset action per row, next to Activity and Delete. Use Activity to see what that account has been doing before you change or remove access. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "User Roles & Permissions Explained · RUAL Documentation" description: "How RUAL access control works: per-user permissions, scopes, settings and user groups mapped to admin, developer, API and visitor roles, with setup tips." canonical: https://docs.rual.nl/cluster/user-roles-explained language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # User Roles & Permissions Explained RUAL does not use fixed roles: access is composed from per-user permission settings, scopes, and optional user groups. Learn how the pieces map to administrators, developers, API users, and visitors. ## The Access Model If you come from a system with fixed roles like "admin" or "editor", RUAL works differently: there are no predefined roles. Instead, what a user can do is composed from three layers on their user document: - **Permissions and settings**: named switches such as `Staging Developer` or `is_root_user` that control what the account may do in RUAL Studio and through cluster APIs. - **Scopes**: values in the `custom_scopes` array that grant access to specific pages and APIs built on the cluster. - **User groups**: reusable bundles of scopes, assigned through the `groups_guid` array, that act like role templates. Mapped to the classic role mental model: - **Cluster administrators** hold the `Cluster Admin` permission and the `is_root_user` setting. In the `All users` list they are marked with a `*` (star) and possess full access to all resources within the cluster. - **Developers** hold blueprint permissions: `Wildcard Scope` to create new resources, plus `Staging Developer` or `Production Developer` depending on how far they may go. - **API-only users** have an account but never open RUAL Studio. They authenticate through the login APIs and their scopes decide which pages and APIs they can call. - **Visitors** have no account at all. They only reach content in the `*public` scope; anything scoped differently redirects them to the login or forbidden page. ## Permission Levels Guide These are the permissions and settings you will assign most often. The full list of fifteen permissions is documented in [User Access Management](https://docs.rual.nl/cluster/user-access-management). | Permission / Setting | What It Allows | Typical Holder | | --- | --- | --- | | `Cluster Admin` | Complete access to all resources within the cluster. | Cluster owners: grant to a select few. | | `is_root_user` | Marks the account as cluster administrator, with extra security measures against login attacks, password forget requests, and sign-in. | Cluster administrators. | | `Wildcard Scope` | Comprehensive access to any custom API and custom page created in RUAL; essential for creating new resources. | Developers. | | `Staging Developer` | View, modify, and simulate blueprints in development. | Junior developers. | | `Production Developer` | View, modify, and deploy blueprints to production. | Senior developers. | | `Blueprints: Production Run` | Controls the `Run: Production` feature in play mode: executing the non-deployed blueprint against production data. | Developers debugging live behavior. | | `Manage User Permissions` (`setting_manage_users`) | Manage other users' permissions and the `allow_login_as_user` setting. Without it, you cannot modify accounts with higher permissions. | Administrators. | | `allow_login_as_user` | Allows the account to be signed in as, using the user signin-as function in blueprints. Cannot be enabled for `is_root_user` accounts. | Support impersonation targets, off by default. | Finer-grained blueprint permissions. `Blueprints: All Access`, `Blueprints: Activate Blueprints`, `Blueprints: Remove Blueprints`, and `Blueprints: Deploy Blueprints`: let you tune exactly how far each developer may go. Cluster APIs use the same model with scope values such as `setting_manage_users` and `setting_edit_blueprints`. ## RUAL Studio Access vs API Access There are two distinct ways to work with a cluster, controlled separately: | Access type | Used for | Authenticated by | | --- | --- | --- | | **RUAL Studio access** | Building blueprints, managing users, storage, and deployment through the browser. | Username and password authorized for RUAL Studio Access. | | **API access** | Calling cluster APIs from custom frontends, scripts, or external services. | An `access_token` obtained through the login APIs. | API tokens are valid for 14 days and extended automatically while actively used. Pass a token as an `Authorization: Bearer` header, as `x-authtoken` or `x-token`, as a cookie named `access_token`, or in the URL as `?access_token=`. See [Getting Access](https://docs.rual.nl/cluster/getting-access) for the onboarding flow and [Cluster APIs](https://docs.rual.nl/cluster/api) for the authentication reference. For API access, the `custom_scopes` array on the user document is decisive: every page and API you build is protected by scopes, and a token only reaches the endpoints its user's scopes cover. An account with the scopes below can call pages and APIs scoped `view_customers` or `create_orders`, and nothing else: ``` { "custom_scopes": [ "view_customers", "create_orders" ], "username": "jane.doe" } ``` ## User Groups as Role Templates User groups are the closest thing RUAL has to roles. A group is a document in the `usergroups` database with a `name` and a `scope` array. Assign a user to a group by adding the group's guid to the `groups_guid` array on the user document: from then on, the user receives every scope in that group. A user can belong to multiple groups and receives all unique scopes from all of them, plus any individual scopes in their own `custom_scopes`. Model your groups after the kinds of users your cluster serves: | Group | Example Scopes | Purpose | | --- | --- | --- | | `Developers` | `Wildcard Scope`, `Staging Developer`, `Blueprints: Activate Blueprints` | Team members who build and maintain blueprints. | | `Customers` | `view_orders`, `create_tickets` | End users who sign in to your app and reach its scoped pages and APIs. | | `Support` | `view_customers`, `view_tickets`, `create_notes` | Staff who help customers without seeing everything. | Visitors need no group: without an account they only see `*public` content. A user assigned to two groups looks like this: ``` { "custom_scopes": [], "groups_guid": [ "94ad734891564818888bea7e2519aedd", "1c5bfaf8fb034f8498d204c5072b554e" ], "username": "jane.doe" } ``` ## Setting Up Team Member Access - Sign in to [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio) with an administrator account and open the `All users` section from the left-side menu. - Create the user with a username and password, and share the cluster URL and credentials privately. - Assign the permissions the work requires: for a developer, start with `Staging Developer` and add more only when needed. - Add any individual scopes to `custom_scopes`, or better: assign the user to a group via `groups_guid` so permissions follow the role, not the person. - Have the user sign in and complete first-time setup, including two-factor authentication when your cluster requires it. ## Security Best Practices - **Least privilege**: grant only the scopes and permissions a user needs, preferably through user groups instead of individual scopes. - **Protect root accounts**: `is_root_user` accounts get extra protection against login attacks and password forget requests. Keep them few, and never enable `allow_login_as_user` for them (RUAL prevents this). - **Guard system settings**: the `Manage System Settings` permission controls who can view and change system settings; access to secured system settings is logged. - **Review the audit log**: sensitive actions such as removing blueprints and changing permissions are recorded: review the log regularly. - **Use two-factor authentication**: users manage their active sessions and multi-factor options on the default `security` page. > **Login-As-User with Caution** The `allow_login_as_user` setting lets an administrator sign in as that user from blueprints: powerful for support, easy to abuse. Only users with `setting_manage_users` can configure it, it cannot be set on `is_root_user` accounts, and you should enable it only where impersonation is truly needed. RUAL also blocks mass multi-authentication attempts from the same IP address and repeated unsuccessful attempts against the same account by default. ## Next Steps - [Getting Access](https://docs.rual.nl/cluster/getting-access): Obtain cluster credentials, sign in for the first time, and set up API tokens. - [User Access Management](https://docs.rual.nl/cluster/user-access-management): The full permission list, user group examples, and account settings reference. - [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control): Scope your pages and APIs: public and logged-in scopes, rate limiting, and throttling. ## Frequently asked **Does RUAL have fixed user roles like admin and editor?** No. RUAL composes access from per-user permission settings, scopes and optional user groups. What people call an administrator, developer, API user or visitor is a combination of those settings, not a fixed role. **What is the difference between RUAL Studio access and API access?** Studio access decides who may open and edit the cluster management interface. API access is granted through scopes on a token and decides which endpoints that token may call. A user can have one without the other. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Access Token Device Information · RUAL Documentation" description: "The access-token .device property: detailed device, browser and application info for requests to your RUAL application." canonical: https://docs.rual.nl/cluster/user-token-info language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Access Token Device Information Access tokens now include detailed device information through the new `.device` property, providing better insights into how users access your application. ## Migration from .agent The `.agent` property has been deprecated and replaced with `.device`. ## Device Properties The `.device` object contains detailed information about the user's device, browser, and application: | Property | Type | Description | | --- | --- | --- | | `device_type` | `string\|null` | Type of device: `"mobile"`, `"desktop"`, `"tablet"`, `"server"`, or `null` | | `device_os` | `string\|null` | Operating system: `"mac"`, `"windows"`, `"linux"`, `"android"`, `"ios"`, `"chromeos"`, `"other"`, or `null` | | `application` | `string\|null` | Application name (e.g., `"chrome-mac"`, `"safari-ios"`) or `null` | | `browser` | `string\|null` | Browser engine: `"chrome"`, `"firefox"`, `"safari"`, `"edge"`, `"electron"`, or `null` | | `browser_version` | `string\|null` | Version of the browser engine (e.g., `"120.0.0.0"`) or `null` | | `device_version` | `string\|null` | Operating system version (e.g., `"10.15.7"`, `"18.3.2"`) or `null` | | `is_mobile` | `boolean` | `true` if the device is a mobile phone | | `is_tablet` | `boolean` | `true` if the device is a tablet | | `is_desktop` | `boolean` | `true` if the device is a desktop or laptop | | `is_bot` | `boolean` | `true` if the request comes from a bot or crawler | | `is_ai` | `boolean` | `true` if the bot is an AI assistant or training bot | | `ai_type` | `string\|null` | Type of AI bot: `"crawl"` (training/research) or `"user"` (user-initiated), or `null` | ## Desktop Browser Examples Here are examples of device information for common desktop browsers: #### Chrome on macOS ``` { "ai_type": null, "application": "chrome-mac", "browser": "chrome", "browser_version": "140.0.0.0", "device_os": "mac", "device_type": "desktop", "device_version": "10.15.7", "is_ai": false, "is_bot": false, "is_desktop": true, "is_mobile": false, "is_tablet": false } ``` #### Safari on macOS ``` { "ai_type": null, "application": "safari-mac", "browser": "safari", "browser_version": "17.0", "device_os": "mac", "device_type": "desktop", "device_version": "10.15.7", "is_ai": false, "is_bot": false, "is_desktop": true, "is_mobile": false, "is_tablet": false } ``` ## Mobile Device Examples Device information for mobile devices provides detailed insights into the user's mobile experience: #### iPhone Safari ``` { "ai_type": null, "application": "safari-ios", "browser": "safari", "device_os": "ios", "device_type": "mobile", "device_version": "18.3.2", "is_ai": false, "is_bot": false, "is_desktop": false, "is_mobile": true, "is_tablet": false } ``` #### Android Chrome ``` { "ai_type": null, "application": "chrome-android", "browser": "chrome", "browser_version": "114.0.0.0", "device_os": "android", "device_type": "mobile", "device_version": "10", "is_ai": false, "is_bot": false, "is_desktop": false, "is_mobile": true, "is_tablet": false } ``` ## Tablet Examples #### iPad ``` { "ai_type": null, "application": "safari-ios", "browser": "safari", "browser_version": "17.0", "device_os": "ios", "device_type": "tablet", "device_version": "17.0", "is_ai": false, "is_bot": false, "is_desktop": false, "is_mobile": false, "is_tablet": true } ``` ## Bot and AI Examples The device information can also identify various types of bots and AI assistants accessing your application: #### Search Engine Bot ``` { "ai_type": null, "application": "googlebot", "browser": "googlebot", "device_os": null, "device_type": "server", "is_ai": false, "is_bot": true, "is_desktop": false, "is_mobile": false, "is_tablet": false } ``` #### AI Assistant (User-Initiated) ``` { "ai_type": "user", "application": "chatgpt-user", "browser": "chatgpt-user", "browser_version": "1.0", "device_os": null, "device_type": "server", "is_ai": true, "is_bot": true, "is_desktop": false, "is_mobile": false, "is_tablet": false } ``` Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "How to deploy · RUAL Documentation" description: "Deploy RUAL blueprints to a cluster." canonical: https://docs.rual.nl/deployment/how-to-deploy language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # How to Deploy Since RUAL version 13.0.7, you can deploy your blueprints and components to production while continuing to operate your development environment as you were accustomed to. ### What is Production In RUAL, the default setting places you within the development environment. Contrary to traditional methods, we have merged the development and production environments into one. Previously, saving a blueprint automatically deployed it to production. Since the release of [version 13.0.7](https://docs.rual.nl/core-versions/130007), we have revised this workflow to create a clearer distinction between development and production environments. Now, saving changes does not automatically trigger deployment; instead, you must navigate to the Menu and select `Deployment`. Here, you will find a list of recently modified blueprints, components, and more that you can choose to deploy. ### Development View By default, using [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio) automatically redirects you to State Pages with the `?development` query. If you are logged in with a RUAL Developer User account, you will see the non-deployed blueprint. Otherwise, you will view the production blueprint, just like any other public user accessing that page. When you select `Production Run` within your blueprint, it executes the non-deployed blueprint, facilitating easier debugging with production data. To prevent RUAL Developers from executing development blueprints too readily, consider disabling the "Run Production" option in their [user accounts](https://docs.rual.nl/cluster/user-access-management#permissions). ### Deployment By default, only cluster owners are permitted to initiate a blueprint deployment. You can compare your currently deployed production blueprint with your development blueprint using the `compare` button. You also have the option to grant 'Allow Deployment' permissions to your RUAL Developers while managing [user access](https://docs.rual.nl/cluster/user-access-management#permissions). ## Frequently asked **How do I deploy a RUAL blueprint to production?** Since RUAL 13.0.7 you deploy blueprints and components to production while continuing to work in the development environment. Your saved work stays in the development view until you deploy it. **Why do my changes not show up for users after saving?** Saving keeps the change in development. Production keeps serving the last deployed version until you deploy again, which is what makes it safe to build and test during working hours. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Build Your First API · RUAL Documentation" description: "Step-by-step RUAL tutorial: register a GET endpoint, search storage, reply in JSON, validate request parameters, secure it with scopes, and test with curl." canonical: https://docs.rual.nl/cluster/api-quickstart language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Build Your First API A step-by-step tutorial: register a GET endpoint, search your storage, reply in JSON, validate request parameters, secure the endpoint with scopes, and test it with curl. In this tutorial you build a real API endpoint: `GET /api/v1/tasks`, which returns the open tasks from the `tasks` storage as JSON. It picks up where the [Quickstart](https://docs.rual.nl/getting-started/quickstart#first-api-endpoint) left off. The quickstart shows the smallest possible endpoint in five minutes; this page goes deeper into URIs, storage queries, validation, and security. If you followed [Build Your First CRUD App](https://docs.rual.nl/getting-started/first-crud-app), you already have the `tasks` storage this endpoint reads from. ## 1. Create the API Blueprint An API endpoint lives in its own blueprint. Open the blueprints overview in [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio), create a new blueprint, and choose the type `Build an API endpoint`. This seeds the canvas with an [`on_startup_register_uri_get`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_get) block. The trigger that starts your flow on every incoming request. Name the blueprint after what it serves, for example `Tasks API`. One blueprint per endpoint (or per small group of related endpoints) keeps flows readable, similar to assigning a single class to a file in traditional coding. ## 2. Define the Endpoint The trigger block carries the endpoint definition. Set the URI field to `v1/tasks`. The block exposes URIs under `/api/`, so your endpoint becomes `GET /api/v1/tasks`. Prefixing with a version like `v1` is a convention, not a requirement, but it lets you introduce `v2` later without breaking existing consumers. The seeded block answers GET requests. Other methods have their own trigger blocks: for example [`on_startup_register_uri_post`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_post) for POST: and a single blueprint can hold several of them when the endpoints belong together. URIs can be multiple segments deep and may include named parameters such as `v1/tasks/:guid`. You will read those parameters in step 5. Keep the URI lowercase and plural, `v1/tasks`, not `v1/getTasks`. The URI is the public contract of your endpoint. ## 3. Create the Handler Function The trigger block does not contain logic itself. It hands each request to a function. Drag from the block's function out-pin into an empty area of the canvas and create a [`trigger_custom_function`](https://docs.rual.nl/block-types/globals%2Cfunction%20execution/trigger_custom_function), naming it `list_tasks`. From then on, the function's `flow` out-pin runs on every request to `GET /api/v1/tasks`. Functions are private to their blueprint by default and can be reused from other flows with [`function_custom_execute`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute): handy when a page flow and an API endpoint need the same query logic. ## 4. Read Data and Reply in JSON Inside the function, search the `tasks` storage and send the results back with [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json). Wire the blocks like this: | Block | Purpose | Connects To | | --- | --- | --- | | `storage` (tasks) | Selects the storage to search | `function_search` (storage pin) | | [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) (status = open) | Match only open tasks | [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) | | [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) | Wrap the term query in an efficient filter context | `query_and` | | [`query_sort_field`](https://docs.rual.nl/block-types/query/query_sort_field) (_meta.created, desc) | Newest tasks first | `query_and` | | `query_and` | Combine filter and sort into one query | `function_search` (query pin) | | `number_default` (50) | Limit the result set | `function_search` (limit pin) | | `function_search` | Runs the search; outputs an `array` of documents | [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) (data pin) | | [`httpconnection_current_request`](https://docs.rual.nl/block-types/http%20connection/httpconnection_current_request) | Reference to the incoming request | [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) (connection pin) | | `number_default` (200) | The HTTP status code | [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) (code pin) | Connect the function's `flow` out-pin to the `flow` in-pin of `function_search`, and from there to [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json). The flow pins control execution order, the data pins carry the results. Query composition is covered in depth under [Building Search Queries](https://docs.rual.nl/blueprints/storage#building-search-queries). ## 5. Validate the Request Real endpoints check their input before touching storage. Say you support an optional `status` parameter in the URI, declared as `v1/tasks/:status`. Read it with [`httpconnection_get_params`](https://docs.rual.nl/block-types/http%20connection/httpconnection_get_params). Connect [`httpconnection_current_request`](https://docs.rual.nl/block-types/http%20connection/httpconnection_current_request) to its connection in-pin, and it outputs a `params` object containing `status`. To make a parameter required, check it before running the search: - Add a [`condition_not_empty_value`](https://docs.rual.nl/block-types/condition/condition_not_empty_value) block and connect the `status` param to it. - Add a `branch` block, connect the function's flow and the condition. It splits execution into `true` and `false` paths. - On the `false` path, reply with a second [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json): a `number_default` of `400` on the code pin and an [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields) with an `error` field such as `missing required parameter: status` on the data pin. - On the `true` path, run the search from step 4: now using the param value in the term query instead of a fixed value. Blocks like `function_search` also expose `success` and `error` out-pins. Use them to return a `500` with a useful message instead of failing silently. When a flow misbehaves, [Debugging](https://docs.rual.nl/troubleshooting/debugging) shows how to trace execution inside the blueprint. ## 6. Scopes and Security New endpoints use the `*public` scope by default: anyone on the internet can call them. Click the lock icon on the trigger block to open the Scopes Management Modal and change that: - `*loggedin`: restrict the endpoint to users with a valid `access_token`. Tokens are issued by the login APIs, are valid for 14 days, and are extended automatically while actively used. - **Custom scopes**. Create your own (for example `tasks-read`) and assign them to only the users or groups that may call this endpoint. Callers can pass a token five ways: an `Authorization: Bearer` header, an `x-authtoken` or `x-token` header, an `access_token` cookie, or `?access_token=` in the query string. The same modal also holds two per-endpoint protection switches: a **rate limit** (key type, max requests, timeframe in seconds: extra requests are blocked) and a **throttle** (key type plus a timeout, allowing one request per interval). Both are explained in detail under [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control). > **Built-in APIs Have Their Own Limits** The cluster's public system APIs carry a fixed default rate limit of `100` requests per second that cannot be changed cluster-wide. Custom endpoints like the one you just built have no such fixed limit. Add a rate limit or throttle per endpoint when you need one. ## 7. Activate and Test APIs are inactive by default. Save the blueprint (`Cmd/Ctrl + S`), then click `Activate` in the top bar. To test the saved, not yet deployed: version, call the endpoint in development view by appending `?development`: ``` curl "https:///api/v1/tasks?development" ``` ``` [ { "_meta": { "cms": 1752768000362, "created": 1752768000, "entity": 1, "expiry": -1, "guid": "b34686f50e1b04964250f13391274d85308c129a793c6ec97bc41870fc67081e", "removed": 0, "ums": 1752768000803, "update_hash": "60e27209fa93063b5605e41605c2722ed428cae0", "updated": 1752768000 }, "owner_guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "owner_name": "joe doe", "status": "open", "title": "write the release notes" } ] ``` A `401` or `403` means the scope check failed: verify the lock icon shows the intended scope and that your token is passed correctly. Consumers only see the endpoint after you [deploy](https://docs.rual.nl/deployment/how-to-deploy) the blueprint to production. ## 8. Next Steps You registered a versioned GET endpoint, wired a storage query into a JSON reply, validated input with a branch, and secured the endpoint. Where to go from here: - [Cluster APIs](https://docs.rual.nl/cluster/api): Your cluster ships 40+ built-in REST APIs for managing users, storage, and blueprints. - [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control): Scopes, the Scopes Management Modal, rate limiting, and throttling in full detail. - [Quickstart](https://docs.rual.nl/getting-started/quickstart): The 15-minute version: Hello World page plus a minimal JSON endpoint. ## Frequently asked **How do I build a REST API endpoint in RUAL?** Register the route with an on_startup_register_uri_get block in its own blueprint, query storage in the flow, and reply with a JSON body. Then save, press Activate, and deploy when you want production traffic to reach it. **How do I secure a custom RUAL API endpoint?** Assign a scope to the endpoint in the Scopes Management Modal. Callers then need a token carrying that scope. You can add a rate limit or a throttle per endpoint in the same modal. **Do custom RUAL endpoints have a rate limit by default?** No. The cluster's built-in public system APIs carry a fixed default of 100 requests per second, but custom endpoints have no fixed limit. Add a rate limit or throttle per endpoint when you need one. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "API Guide: Working with RUAL APIs · RUAL Documentation" description: "Public vs private APIs, token authentication, request/response conventions, error codes, rate limits, and common workflows with curl examples." canonical: https://docs.rual.nl/cluster/api-guide language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # API Guide: Working with RUAL APIs Public vs private APIs, token authentication, request and response conventions, error codes, rate limits, and the workflows you will use daily, with curl examples. Every RUAL cluster exposes REST APIs under `/api/v1`: the built-in [Cluster APIs](https://docs.rual.nl/cluster/api) (blueprints, users, storages, system operations) plus any custom API you build with an [API blueprint](https://docs.rual.nl/cluster/api-quickstart). This guide covers how to call them correctly; the [API reference](https://docs.rual.nl/cluster/api) lists every endpoint with its parameters and example responses. ## Public vs Private APIs | | Public (`*public`) | Private (`*loggedin` / custom scopes) | | --- | --- | --- | | Authentication | None: anyone with the URL can call it | A valid `access_token` is required | | Default for | Built-in cluster APIs marked public | Everything else; your own APIs unless you open them | | Rate limit | 100 requests/second per endpoint by default | Per-user; configurable per API | | Typical use | Status pages, webhooks, public content | Anything reading or writing cluster data | You set the audience per API block with its lock icon: `*public`, `*loggedin` (any valid token), or a custom scope you invent (e.g. `billing_read`) and then grant to selected users. See [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control). ## Authentication API calls authenticate with an `access_token`, obtained through the sign-in APIs (or via [RUAL Studio](https://docs.rual.nl/cluster/getting-access)). Tokens are valid for **14 days** and automatically extend while actively used. You do not need to re-login during normal operation. The token is accepted in five equivalent locations, pick one: | Location | Example | | --- | --- | | `Authorization` header (recommended) | `Authorization: Bearer eyJhbGciOi...` | | `x-authtoken` header | `x-authtoken: eyJhbGciOi...` | | `x-token` header | `x-token: eyJhbGciOi...` | | Cookie | `Cookie: access_token=eyJhbGciOi...` | | Query parameter | `?access_token=eyJhbGciOi...` (handy for browser downloads) | What a token may do is decided by the user's scopes and the token's `custom_scopes`. See [User Roles & Permissions Explained](https://docs.rual.nl/cluster/user-roles-explained). On clusters with two-factor authentication enforced, sign-in additionally requires the current 2FA code. ## Request Conventions - **Base URL**, `https:// /api/v1`. All endpoints live under it. - **Content type**: send and receive `application/json` unless an endpoint documents otherwise (file upload uses `multipart/form-data`). - **Search endpoints**: accept Elasticsearch-style queries: `term` (exact match), `terms` (one of many), `match` (full-text), `range` (dates/numbers), plus `limit` and `offset` for pagination. - **Documents**. Every stored object carries a `_meta` object with `guid`, `created`, `updated`, and removal state. ## Error Responses Errors return a JSON object with the HTTP status, a machine-readable code and a message: ``` { "error": "DOCUMENT_NOT_FOUND", "message": "The requested document does not exist." } ``` | Status | Meaning | First thing to check | | --- | --- | --- | | `400` | Bad request: invalid body, params, or file | Validate your JSON against the endpoint's parameter table. | | `401` | Unauthenticated: missing/expired token, or 2FA required | Is the token present in an accepted location and not older than 14 days idle? | | `403` | Forbidden: token valid, scope insufficient | The user's scopes vs the endpoint's required scope ([scoping](https://docs.rual.nl/blueprints/remote-access-control)). | | `404` | Not found: endpoint or document does not exist | Typo in the path; for custom APIs: is the blueprint [activated](https://docs.rual.nl/deployment/how-to-deploy)? | | `409` | Conflict: version/update-hash mismatch | Re-read the document and retry with the current `update_hash`. | | `422` | Unprocessable: validation failed | The response body names the failing field/rule. | | `429` | Rate limited | Back off and retry with jitter; see rate limits below. | | `500` | Server error | Retry once; if persistent, check the cluster logs in RUAL Studio. | ## Rate Limiting Public APIs default to **100 requests per second** per endpoint. Custom APIs have their own configurable rate limit and throttle (lock & wait) fields: documented per block in [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control). When you receive a `429`, retry with exponential backoff and jitter; do not retry in a tight loop. ## Common API Workflows #### 1. Sign in to get a token ``` curl -X POST "https://your-cluster.rual.dev/api/v1/auth/signin" \ -H "Content-Type: application/json" \ -d '{"username": "you@example.com", "password": "..."}' ``` The response contains your `access_token`: reuse it for subsequent calls. #### 2. Search documents in a storage ``` curl -X POST "https://your-cluster.rual.dev/api/v1/_system/storages/search" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer eyJhbGciOi..." \ -d '{"limit": 10}' ``` #### 3. Call a custom blueprint API ``` curl "https://your-cluster.rual.dev/api/v1/tasks?access_token=eyJhbGciOi..." ``` Your own endpoints registered with an [`on api get`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_get) block behave like any other API on the cluster: same auth, same error shapes. See [Build Your First API](https://docs.rual.nl/cluster/api-quickstart). ## Quick Reference | Endpoint | Method | Purpose | | --- | --- | --- | | `/api/v1/auth/signin` | POST | Sign in, receive an access token | | `/api/v1/auth/signout` | POST | Invalidate the current token | | `/api/v1/users/search` | POST | List and search cluster users | | `/api/v1/blueprints/search` | POST | List blueprints on the cluster | | `/api/v1/blueprints/update/set/active/{guid}` | PUT | Activate a blueprint (deploy) | | `/api/v1/blueprintactions/search` | POST | Read the blocks inside a blueprint | | `/api/v1/_system/storages/search` | POST | List storage collections | | `/api/v1/_system/assets/search` | POST | Search uploaded assets | | `/api/v1/_system/systemlogs/search` | POST | Read cluster system logs | | `/api/v1/info` | GET | Cluster version, flags, and host info (public) | The full list with per-endpoint parameters, example responses, and required scopes lives in the [Cluster API reference](https://docs.rual.nl/cluster/api). ## Next Steps - [Build Your First API](https://docs.rual.nl/cluster/api-quickstart): Register an endpoint, reply in JSON, validate params, test with curl. - [Cluster API Reference](https://docs.rual.nl/cluster/api): Every endpoint with parameters, examples, and required scopes. - [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control): Scopes, rate limits, and throttling for your own APIs. ## Frequently asked **How do I authenticate against the RUAL API?** Sign in through POST /api/v1/auth/signin to receive an access_token, then send it on every call, preferably as an Authorization: Bearer header. Tokens are valid for 14 days and extend automatically while actively used. **What is the difference between public and private RUAL APIs?** Public APIs (*public) need no authentication and default to 100 requests per second per endpoint. Private APIs require a valid access token, are rate limited per user, and have configurable limits per API. Your own APIs are private unless you open them. **What do RUAL API error responses look like?** A JSON object with the HTTP status, a machine-readable code and a message, for example {"error": "DOCUMENT_NOT_FOUND", "message": "The requested document does not exist."}. A 429 means rate limited: retry with exponential backoff and jitter, never in a tight loop. **How do RUAL search endpoints work?** They accept Elasticsearch-style queries: term for exact match, terms for one of many, match for full-text and range for dates and numbers, plus limit and offset for pagination. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Cluster - API Categories · RUAL Documentation" description: "Browse API categories to find the endpoints you need. Each category contains related API endpoints organized by functionality." canonical: https://docs.rual.nl/cluster/api language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # API Categories Browse API categories to find the endpoints you need. Each category contains related API endpoints organized by functionality, making it easy to discover and integrate with cluster services. ## Authentication Authentication is required for most of our APIs. You'll need to use the `access_token` provided after successful authentication to access these APIs. It's important to note that you cannot obtain the `access_token` directly within blueprints. Users must log in through the appropriate APIs within your application to obtain a token. ## Token Expiry In order to streamline the user experience, we have automated the token expiry process. By default, tokens are valid for 14 days and are automatically extended when actively used. When you make API requests, access pages, or interact with modals created in blueprints, we will automatically update the user's last activity, last token activity, and token expiry. To extend the token's validity, multiple requests need to be made using the token. ## Public API Rate Limiting Some of our APIs are public and come with default rate-limiting restrictions. These APIs have a default rate limit of `100` requests per second. It's not possible to modify the rate-limiting behavior for these APIs on a cluster level. However, if you encounter rate-limiting issues and cannot address them through conventional means, you have the option to build your own custom APIs within blueprinting to circumvent these limitations. ## API Content-Type When making requests to system or cluster APIs, it is crucial to include a `Content-Type` in your request headers. For most APIs, the `Content-Type` should be `application/json`. However, for File API endpoints, you should use `multipart/form-data` and specify `Accept: application/json`. ## Adding Tokens to your requests Once you have obtained an `access_token` through the appropriate APIs, you need to include this token in your request. You can achieve this in several ways, as outlined below: - In the header, you can pass the token as `Authorization` using the `Bearer` scheme. - In the header, you can pass the token as `x-authtoken`. - In the header, you can pass the token as `x-token`. - In the header, you can pass the token as a `cookie` with the name `access_token`. - In the URL, you can pass the token as `?access_token=` in the query string. ### API Categories - [API History](https://docs.rual.nl/cluster/api/api-history): Contains `2` API endpoints. API History related API endpoints - [Audit Logs](https://docs.rual.nl/cluster/api/audit-logs): Contains `2` API endpoints. Audit Logs related API endpoints - [Authentication](https://docs.rual.nl/cluster/api/authentication): Contains `13` API endpoints. Authentication related API endpoints - [Authentication Groups](https://docs.rual.nl/cluster/api/authentication-groups): Contains `5` API endpoints. Authentication Groups related API endpoints - [Blueprint Action Types](https://docs.rual.nl/cluster/api/blueprint-action-types): Contains `3` API endpoints. Blueprint Action Types related API endpoints - [Blueprint Actions](https://docs.rual.nl/cluster/api/blueprint-actions): Contains `16` API endpoints. Blueprint Actions related API endpoints - [Blueprint Activity](https://docs.rual.nl/cluster/api/blueprint-activity): Contains `2` API endpoints. Blueprint Activity related API endpoints - [Blueprint Changes](https://docs.rual.nl/cluster/api/blueprint-changes): Contains `2` API endpoints. Blueprint Changes related API endpoints - [Blueprint Comments](https://docs.rual.nl/cluster/api/blueprint-comments): Contains `3` API endpoints. Blueprint Comments related API endpoints - [Blueprint Components](https://docs.rual.nl/cluster/api/blueprint-components): Contains `5` API endpoints. Blueprint Components related API endpoints - [Blueprint Events](https://docs.rual.nl/cluster/api/blueprint-events): Contains `4` API endpoints. Blueprint Events related API endpoints - [Blueprint Fields](https://docs.rual.nl/cluster/api/blueprint-fields): Contains `5` API endpoints. Blueprint Fields related API endpoints - [Blueprint Flows](https://docs.rual.nl/cluster/api/blueprint-flows): Contains `1` API endpoints. Blueprint Flows related API endpoints - [Blueprint Tags](https://docs.rual.nl/cluster/api/blueprint-tags): Contains `6` API endpoints. Blueprint Tags related API endpoints - [Blueprint Templates](https://docs.rual.nl/cluster/api/blueprint-templates): Contains `11` API endpoints. Blueprint Templates related API endpoints - [Blueprint Validation](https://docs.rual.nl/cluster/api/blueprint-validation): Contains `4` API endpoints. Blueprint Validation related API endpoints - [Blueprints](https://docs.rual.nl/cluster/api/blueprints): Contains `23` API endpoints. Blueprints related API endpoints - [Cache Management](https://docs.rual.nl/cluster/api/cache-management): Contains `4` API endpoints. Cache Management related API endpoints - [Cluster Statistics](https://docs.rual.nl/cluster/api/cluster-statistics): Contains `4` API endpoints. Cluster Statistics related API endpoints - [Entity Management](https://docs.rual.nl/cluster/api/entity-management): Contains `6` API endpoints. Entity Management related API endpoints - [Feedback](https://docs.rual.nl/cluster/api/feedback): Contains `1` API endpoints. Feedback related API endpoints - [IP Management](https://docs.rual.nl/cluster/api/ip-management): Contains `2` API endpoints. IP Management related API endpoints - [Language](https://docs.rual.nl/cluster/api/language): Contains `1` API endpoints. Language related API endpoints - [Language Management](https://docs.rual.nl/cluster/api/language-management): Contains `8` API endpoints. Language Management related API endpoints - [Languages](https://docs.rual.nl/cluster/api/languages): Contains `2` API endpoints. Languages related API endpoints - [Local Devices](https://docs.rual.nl/cluster/api/local-devices): Contains `5` API endpoints. Local Devices related API endpoints - [Node Settings](https://docs.rual.nl/cluster/api/node-settings): Contains `3` API endpoints. Node Settings related API endpoints - [Password Reset](https://docs.rual.nl/cluster/api/password-reset): Contains `2` API endpoints. Password Reset related API endpoints - [Queue](https://docs.rual.nl/cluster/api/queue): Contains `4` API endpoints. Queue related API endpoints - [React Components](https://docs.rual.nl/cluster/api/react-components): Contains `5` API endpoints. React Components related API endpoints - [Redis Keys](https://docs.rual.nl/cluster/api/redis-keys): Contains `8` API endpoints. Redis Keys related API endpoints - [Settings](https://docs.rual.nl/cluster/api/settings): Contains `5` API endpoints. Settings related API endpoints - [Storage](https://docs.rual.nl/cluster/api/storage): Contains `7` API endpoints. Storage related API endpoints - [Storage Events](https://docs.rual.nl/cluster/api/storage-events): Contains `1` API endpoints. Storage Events related API endpoints - [System](https://docs.rual.nl/cluster/api/system): Contains `8` API endpoints. System related API endpoints - [System - Languages](https://docs.rual.nl/cluster/api/system---languages): Contains `1` API endpoints. System - Languages related API endpoints - [System Assets](https://docs.rual.nl/cluster/api/system-assets): Contains `3` API endpoints. System Assets related API endpoints - [System Hints](https://docs.rual.nl/cluster/api/system-hints): Contains `4` API endpoints. System Hints related API endpoints - [System Information](https://docs.rual.nl/cluster/api/system-information): Contains `1` API endpoints. System Information related API endpoints - [System Key-Value](https://docs.rual.nl/cluster/api/system-key-value): Contains `4` API endpoints. System Key-Value related API endpoints - [System Logs](https://docs.rual.nl/cluster/api/system-logs): Contains `2` API endpoints. System Logs related API endpoints - [System Nodes](https://docs.rual.nl/cluster/api/system-nodes): Contains `1` API endpoints. System Nodes related API endpoints - [System Security](https://docs.rual.nl/cluster/api/system-security): Contains `9` API endpoints. System Security related API endpoints - [System Storages](https://docs.rual.nl/cluster/api/system-storages): Contains `5` API endpoints. System Storages related API endpoints - [System Threads](https://docs.rual.nl/cluster/api/system-threads): Contains `2` API endpoints. System Threads related API endpoints - [System Upgrades](https://docs.rual.nl/cluster/api/system-upgrades): Contains `3` API endpoints. System Upgrades related API endpoints - [System Users](https://docs.rual.nl/cluster/api/system-users): Contains `1` API endpoints. System Users related API endpoints - [User Activity](https://docs.rual.nl/cluster/api/user-activity): Contains `2` API endpoints. User Activity related API endpoints - [User Groups](https://docs.rual.nl/cluster/api/user-groups): Contains `9` API endpoints. User Groups related API endpoints - [User Logs](https://docs.rual.nl/cluster/api/user-logs): Contains `2` API endpoints. User Logs related API endpoints - [Users](https://docs.rual.nl/cluster/api/users): Contains `12` API endpoints. Users related API endpoints - [Well-Known](https://docs.rual.nl/cluster/api/well-known): Contains `3` API endpoints. Well-Known related API endpoints ## Frequently asked **Where do I find the RUAL cluster API reference?** The API Categories page lists every cluster API endpoint grouped by category, with parameters, example responses and required scopes per endpoint. Most APIs require an access_token obtained through sign-in; you cannot obtain one from within blueprints. **How do I pass my access token to RUAL APIs?** Five equivalent ways: the Authorization: Bearer header, the x-authtoken or x-token header, an access_token cookie, or the ?access_token= query parameter, which is handy for browser downloads. **What is the rate limit of the public RUAL APIs?** Public APIs carry a default rate limit of 100 requests per second, and that cannot be modified on a cluster level. If you hit that limit and cannot address it conventionally, you can build your own custom APIs within blueprinting to circumvent it. **Which Content-Type do RUAL APIs expect?** application/json for most APIs. File API endpoints use multipart/form-data and should specify Accept: application/json. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Blueprints · RUAL Documentation" description: "The visual programming model: how flows execute, how storage works, and the patterns that keep large blueprints readable." canonical: https://docs.rual.nl/blueprints language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Blueprints The visual programming model: how flows execute, how storage works, and the patterns that keep large blueprints readable. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Introduction · RUAL Documentation" description: "The canvas, blocks, pins and flow: the vocabulary the rest of these pages assume." canonical: https://docs.rual.nl/blueprints/introduction language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Introduction to blueprinting Blueprinting is a collaborative process where multiple users can design and modify workflows on a shared canvas. It allows for the creation and organization of various elements like pages, modals, and APIs, streamlining development and team coordination. ## About Blueprinting A blueprint serves as an expansive canvas for designing workflows, where various blocks can be arranged to create distinct flows. Each blueprint is typically dedicated to a specific component, such as a page, modal, API, or a set of (public) functions, mirroring the practice of assigning a single class to a file in traditional coding. These blueprints can encapsulate diverse elements, from state pages to scheduled events invoking functions from other blueprints. Blueprinting supports collaborative development, allowing multiple users to work on the same blueprint simultaneously. Team members can enhance clarity by adding comments, grouping blocks with custom titles, and copying blocks between blueprints as needed. Furthermore, RUAL ensures comprehensive audit logging of all activities, including blueprint modifications and block interactions, providing an automatic and continuous record of changes, like git commits, but with real-time collaboration. ## Creating a Blueprint When you create a new blueprint, RUAL first asks *what type of blueprint you would like*. Picking a type pre-arranges the canvas with the right starting blocks for that purpose, so you don't have to wire everything up from scratch. The available types are: | Type | Generates | Pick it when | | --- | --- | --- | | `Build a UI Page` | Public or internal webpages (seeds a `state_page` block) | Anything a user opens in the browser | | `Build a UI Modal` | Custom UI modals by name | Dialogs and overlays on top of a page | | `Build a Function` | Custom functions | Reusable logic shared between blueprints | | `Build an API endpoint` | Custom APIs and responses (seeds an `on api` block) | REST endpoints for apps or integrations | | `Build an API Form` | An internal or public API | API-driven forms | | `Database event` | Automation on document create/update | Reactions to storage changes (sync, notifications) | | `Time based event` | Fire and repeat automation by time | Schedules: digests, polling, cleanup | The chosen type only seeds the canvas; a blueprint remains a free-form canvas, so you can always add, remove, or combine blocks of any kind afterwards. Not sure which architecture fits? See [Choosing Your Approach](https://docs.rual.nl/architecture/choosing-approaches). ## Connecting Blocks Blocks connect through **pins**: `flow` pins decide what runs after what, `value/object/array` pins pass data, and special pins (storage, httpconnection, state) carry their resource. Drag from an out-pin to a matching in-pin: or drag onto empty canvas and pick from the search results to create the target block with the connection already made. Every pin type and its colors is documented in [Core Concepts](https://docs.rual.nl/getting-started/core-concepts), and the connection shortcuts (smart pins, distant connections, auto-created blocks) in [Tips & Tricks](https://docs.rual.nl/blueprints/tips-and-tricks). ## Navigational Techniques To streamline navigation within blueprints, you can employ the following methods: - Utilize your mouse scroll wheel to zoom in and out of the current location. - Use your trackpad, if available, to pinch and zoom within the blueprint, which is particularly efficient on devices like a Macbook Pro. These techniques facilitate smooth navigation and exploration of your blueprint. ## Context Menu The default context-menu in [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio), accessible via `right-click` within a blueprint, offers several options for enhancing your workflow: - `Add new block`: This option allows you to quickly create any block at any location, regardless of the pin type. Simply type the name of the block you need after selecting this option. - `Add function`: This adds a new, [blank function](https://docs.rual.nl/block-types/globals%2Cfunction%20execution/trigger_custom_function) block, enabling the easy creation of functions anywhere in the blueprint. Functions are private by default. - `User guid`: This generates a unique identifier (GUID) for the current user, allowing it to be used elsewhere in the blueprint. - `Go to center`: This centers your view on the blueprint. You might need to zoom out if the view is still zoomed in. - `Add marker`: This places a marker at your current position, useful for collaborative work in the same blueprint. A sound plays upon adding a marker. - `Add warning`: This allows you to add a warning sign to prevent unintended changes when collaborating on a blueprint. - `Add comment`: This allows you to [add a comment](https://docs.rual.nl/block-types/studio%20functions/comment_chat) at your current position. You can type your comment directly into the newly added block. - `Reload`: This refreshes the current blueprint view, useful for resolving display issues that sometimes occur due to browser behaviors like auto-input focus. ## Top bar: options At the top right side of your blueprint, you'll find some options to make your life easier: - `Explorer`: This button simplifies finding different types of blocks within your blueprint. For instance, it groups all API blocks for easy navigation. - `Tags`: Manage and create new tags with this button, allowing you to quickly understand a blueprint's purpose from the overview. - `Feedback`: Share suggestions or report issues about the blueprint directly with the team. - `Activate`: When visible, this button activates the APIs, modals, and pages within the blueprint. By default, these elements are inactive until you hit `activate`. - `Deactivate`: When visible, this button deactivates the APIs, modals, and pages within the blueprint. Confirmation is required, and the action is audit-logged. ## Bottom bar The bottom bar gathers everything related to your unsaved changes into a single unified bar at the bottom of your blueprint. Its buttons become active as you make changes: - `undo`: Steps back through your unsaved edits one change at a time. - `redo`: Re-applies a change you just undid. - `console`: Opens the console for the current blueprint. - `changes`: Shows how many unsaved changes are open. Expand it to sequentially review all open unsaved blocks and their respective changes. - `Reset`: Discards all unsaved modifications, restoring the blueprint to its most recent saved state, including changes made by other collaborators. - `Save`: Saves your current modifications. You could also use `Cmd/Ctrl + s` to save. Saving renders the undo option unavailable, generates a new [precompiled](https://docs.rual.nl/blueprints/precompiled) version of the blueprint, and initiates [deployment](https://docs.rual.nl/blueprints/tips-and-tricks#saving-blueprint) if the blueprint includes State UI pages, APIs, time-based events, or database events. ## Frequently asked **What are the blueprint types in RUAL?** When you create a blueprint, RUAL asks what type you would like: Build a UI Page, Build a UI Modal, Build a Function, Build an API endpoint, Build an API Form, Database event or Time based event. Each type pre-arranges the canvas with the right starting blocks for that job, and the canvas stays free-form afterwards. **Can multiple people work on the same RUAL blueprint?** Yes. Blueprinting supports real-time collaboration on a shared canvas, with comments, titled groups, markers and warnings to coordinate, and every modification is recorded in the audit log automatically. **How do I undo a change in a RUAL blueprint?** The bottom bar gathers your unsaved changes: undo steps back one edit at a time, the changes counter expands into a per-block review, and Reset discards all unsaved modifications back to the most recent saved state, including changes made by collaborators. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Tips & tricks · RUAL Documentation" description: "Navigate big blueprints, group and copy blocks, and use the search menu efficiently." canonical: https://docs.rual.nl/blueprints/tips-and-tricks language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Tips and tricks Discover some valuable tips and tricks for efficient blueprinting, designed to help you navigate and work with blueprints more effectively. ## Establishing Block Connections When you wish to connect two blocks, you can achieve this by simply `left-clicking` on the output pin and dragging it into the input pin, or vice versa. Additionally, connections can be established automatically when you drag out of a pin and create a new block directly from the search results that appear when you release the pin in the empty area of the blueprint. The system will attempt to provide you with blocks that are similar in type to the pin you initially dragged out. ## Automatic Block Creation For increased efficiency in blueprint workflows, certain blocks are generated automatically when you perform a `right-click` drag action from an in-pin or out-pin to the blueprint background. This feature is primarily employed with value pins, especially in conjunction with the [`custom value block`](https://docs.rual.nl/block-types/value/value_default). Here is a list of blocks that are automatically generated when you perform a `right-click drag`: | Pin Type | Block Created | | --- | --- | | `state` | [state_text](https://docs.rual.nl/block-types/state%20ui/state_text) | | `file` | [file_custom](https://docs.rual.nl/block-types/files/file_custom) | | `date` | [date_currentdate](https://docs.rual.nl/block-types/date%2Ctimezone/date_currentdate) | | `object` | [object_new_fields](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields) (in-pin), [object_update_fields](https://docs.rual.nl/block-types/object%2Cfields/object_update_fields) (out-pin) | | `value` | [value_default](https://docs.rual.nl/block-types/value/value_default) | | `uiclass` | [uiclass_custom](https://docs.rual.nl/block-types/ui%20class/uiclass_custom) | | `uistyle` | [uistyle_custom](https://docs.rual.nl/block-types/styling/uistyle_custom) | | `fielddefinitions` | [fielddefinitions_get_multiple](https://docs.rual.nl/block-types/fields/fielddefinitions_get_multiple) | | `array` | [array_values_default](https://docs.rual.nl/block-types/array/array_values_default) | | `storage` | [storage](https://docs.rual.nl/block-types/globals%2Cstorage/storage) | | `user` | [user_current](https://docs.rual.nl/block-types/users%2Cglobals/user_current) | | `flow` | [function_custom_execute](https://docs.rual.nl/block-types/function%20execution/function_custom_execute) (out-pin) | | `condition` | [condition_is_true](https://docs.rual.nl/block-types/condition/condition_is_true) | | `inpinfunction` | [inpinfunction_select](https://docs.rual.nl/block-types/modal%20functions/inpinfunction_select) | ## Smart Pin Reveal To keep blueprints uncluttered, blocks don't always show every pin they support — many keep their less-common in-pins hidden until they're actually needed. When you start dragging a connection out of an out-pin, every block on the canvas inspects the type you're carrying and **automatically reveals any hidden in-pin that can accept it**. Just drag from the out-pin of one block toward another, and the matching in-pin pops up so you can drop the connection straight onto it — there's no need to add the pin by hand first. As you drag, the connection wire is colour-coded to the pin's type (for example `value`, `array`, or `condition`) and a small label shows the pin's name and type. In-pins of incompatible types stay hidden, so you only ever see the connections that are actually valid. ## Selecting blocks You can select any block within the blueprint by clicking on its header area, which is typically the darker-colored part of the block. Once selected, an orange border will appear around the block, indicating your selection. To select multiple blocks at once, you can hold down `Cmd/Ctrl` and click on another unselected block. Clicking on a block that's already selected will deselect it. Alternatively, you can select multiple blocks by `right-clicking` and dragging to create a dark selection area. Simply extend this area over the blocks you wish to select. ## Connecting Distant Blocks Occasionally, the block you wish to connect to may be located outside your current viewport, necessitating a shift in your view. It's not possible to drag a connection directly from the current pin to the distant one. To address this scenario, we've introduced the double-click option. Simply double-click on the pin you intend to connect, pan to the new block, and click once on the target pin. This sequence establishes the connection. ## Disconnecting blocks If you need to disconnect two or more blocks that are linked, you can achieve this by simply `right-clicking` on the relevant pin. This action will sever the connection between the blocks on both ends. In cases where a pin has multiple connections, a modal will prompt you for confirmation before proceeding. ## Removing a block After selecting one or multiple blocks, you can remove them by pressing `backspace` on your keyboard. The block will turn opaque and be outlined with a red dashed border, indicating it has been removed but not yet permanently deleted. To finalize the deletion, simply save the blueprint using the `save your changes` button, after which the block will permanently disappear. ## Staging and saving a blueprint After making modifications in your blueprint, you can save them using the `save your changes` button. However, if you wish to quickly toggle between the blueprint and your pages to view these changes, you can append `?staging` to the URL of the page you're reviewing. This allows you to see the changes without having to save each one. Alternatively, you can use the shortcut `Cmd/Ctrl` + `S` to save and deploy your changes in the blueprint swiftly. ## Block options Each block in the blueprint comes with its own set of options. By clicking on the three dots at the top of a block, a menu will appear, offering block-specific functionalities. However, the following options are always included: - `Modify name`: This feature enables you to change the block's name, facilitating easier documentation of a particular flow without needing to use a comment block. - `Revisions`: Selecting this option opens the logbook, but it's limited to the audit history of that specific block, allowing you to quickly review recent changes affecting it. - `Documentation`: This opens a new tab with the documentation specific to that block. ## Edit pin sorting Blocks that expose several pins of the same type — such as the multiple out-pins on a `private function` block — let you control the order in which those pins appear. Open the block's options menu via the three dots and choose `Edit pin sorting` to bring up the sorting popup. In the popup, each pin is listed with its current position number. Grab the handle on the left of any row and drag it up or down to reorder the pins; the numbering updates as you go. Once you save the blueprint, the pins on the block are rearranged to match the new order, keeping busy blocks tidy and making their connections easier to follow. ## Simulating Functions Certain blocks, particularly those at the beginning of a flow, feature a play icon. Clicking this icon triggers a popup where you can switch between different play modes, activate the current mode (default is 'simulation'), and adjust simulation values. For instance, in the example below, `key` represents an out-pin whose value can be modified. During simulation, this value is passed to the connected blocks. The simulation mode is particularly useful for thorough end-stage debugging. When configuring simulation values, it's essential to use the correct format based on the data type. The table below provides the proper formatting syntax for each type, ensuring your simulations run accurately and produce expected results: | Pin Type | Formatting | | --- | --- | | `date` | `1749321690` (timestamp as number) | | `number` | `1` | | `object` | `{"key": "value}` or `{}` | | `value` | `"value"` or `""` | | `uiclass` | `"uiclass"` or `""` | | `uiattributes` | `{"loading": "lazy"}` or `{}` | | `uistyle` | `{"background": "grey"}` or `{}` | | `array` | `["value1", "value2"]` or `[]` | | `condition` | `true` or `false` | ## Blueprint namespacing When managing numerous functions used across different blueprints, it's advisable to organize them with namespaces. Instead of manually prefixing each function, we recommend utilizing the built-in namespace feature. By clicking on the blueprint's title at the top, a popup will appear, allowing you to add a namespace. Entering a namespace here will automatically prepend it to all existing functions, both public and private, within the blueprint. Consequently, when these functions are referenced in other blueprints, they will display with the specified namespace prefix, clearly indicating their origin blueprint for easier identification and organization. ## Blueprint block searching You can press `Ctrl + F` to open the search bar at the bottom left of your screen. This makes it easy to quickly locate blocks or values within the cluster you're working in, saving you from endlessly scrolling to find a specific block. Clicking on a search result will automatically take you to the desired block within your blueprint. This feature is especially helpful when you're working in a blueprint you're not very familiar with. This works for most block types. Except for translated data, which unfortunately aren't searchable. ## Minimap For larger blueprints, a `minimap` is available in the bottom-right corner of the screen, giving you a bird's-eye view of the entire blueprint. Every block is plotted on it as a small marker, and the highlighted rectangle represents your current viewport — so you can see at a glance where you are within the wider blueprint and how much surrounds you off-screen. The minimap is resizable, so you can drag it to whatever dimensions suit your workflow, making it easy to track your position and navigate around even the most sprawling blueprints. ## Testing and collaboration habits A few habits that pay off once blueprints get serious: - **Simulate before you activate**. Every trigger block has a play button: run the flow with simulation values and inspect each out-pin before it ever sees production. Full technique in [Debugging Blueprints](https://docs.rual.nl/troubleshooting/debugging). - **Comment as you go**: right-click → Add comment on anything non-obvious; future-you and your teammates will thank you. Warnings mark do-not-touch areas; markers ping collaborators to a spot. - **Review changes before saving**. The `changes` counter in the bottom bar expands into a per-block diff; check it the way you'd check a git diff before committing. - **Use the Timeline for archaeology**. The blueprint Timeline replays every save with author and timestamp; block-level Revisions give the same for a single block (see [audit log](https://docs.rual.nl/troubleshooting/debugging#audit-log)). - **Split when two people fight over one canvas**: functions in a shared blueprint, per-feature blueprints elsewhere (see [Best Practices](https://docs.rual.nl/blueprints/best-practices)). ## Frequently asked **How do I connect two blocks that are far apart in a RUAL blueprint?** Double-click the pin you want to connect, pan to the other block, and click once on the target pin to establish the connection. Dragging a connection directly to a block outside the current viewport is not possible. **How do I search for a block inside a RUAL blueprint?** Press Ctrl + F to open the search bar at the bottom left of the screen. Clicking a result takes you straight to that block. It works for most block types, except translated data, which is not searchable. **How do I preview unsaved blueprint changes in RUAL?** Append ?staging to the URL of the page you are reviewing to see your unsaved changes without saving each one. Cmd/Ctrl + S saves the blueprint when you are satisfied. **How do I simulate a RUAL flow before activating it?** Blocks at the start of a flow carry a play icon. Click it to open the play mode popup, set simulation values per out-pin in the right format for its data type, and run the flow to inspect what each connected block receives. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Best Practices · RUAL Documentation" description: "Naming, namespaces, tags and structure that keep a blueprint maintainable." canonical: https://docs.rual.nl/blueprints/best-practices language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Blueprint Best Practices Opinionated guidance for keeping blueprints small, fast, and maintainable: from naming and functions to caching, deployment discipline, and teamwork. RUAL gives you a free-form canvas, and that freedom cuts both ways: a blueprint can grow into an unnavigable tangle just as easily as into a clean component. The practices below are the conventions we recommend for teams building serious systems. Each one links back to the feature documentation it builds on. ## When to Split Blueprints Dedicate each blueprint to a single component: a page, a modal, an API endpoint, or a set of related functions. This mirrors the traditional practice of assigning one class to one file. Resist the urge to keep adding "just one more flow" to an existing canvas. Signs a blueprint has grown too big: - You rely on the [minimap](https://docs.rual.nl/blueprints/tips-and-tricks#minimap) and `Ctrl + F` [block search](https://docs.rual.nl/blueprints/tips-and-tricks#blueprint-block-searches) for every navigation, because scrolling no longer works. - The blueprint mixes unrelated responsibilities. A page, its API, and three scheduled jobs living side by side. - New team members cannot tell what the blueprint *is* from the overview, even with [tags](https://docs.rual.nl/blueprints/introduction#top-bar-options). When you do create a new blueprint, pick the matching [blueprint type](https://docs.rual.nl/blueprints/introduction#creating) so the canvas starts with the right blocks for a page, modal, API, function, or event. Splitting does not mean duplicating logic. Share logic between blueprints with **public functions**: create a [`trigger_custom_function`](https://docs.rual.nl/block-types/globals%2Cfunction%20execution/trigger_custom_function) block in a dedicated function blueprint and call it from anywhere else with [`function_custom_execute`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute). Functions are private by default. Make them public only when another blueprint genuinely needs them. See [Block Execution](https://docs.rual.nl/blueprints/block-execution) for how function calls execute. ## Naming Conventions Names are your primary documentation in a system with no code files. Adopt conventions early and apply them everywhere: | Artifact | Convention | Example | | --- | --- | --- | | Blueprint | Name it after the single component it implements. | `Task list page`, `Tasks API` | | Namespace | Set one namespace per function blueprint; it prefixes all functions, public and private. | `tasks` → `tasks.list_tasks` | | Function event | snake_case verb + noun describing the action. | `list_tasks`, `send_invoice_email` | | Storage | Lowercase plural noun: field names are stored lowercase and searches are case-sensitive, so lowercase avoids a whole class of bugs. | `tasks`, `messages` | Use the built-in [namespace feature](https://docs.rual.nl/blueprints/tips-and-tricks#blueprint-namespace) (click the blueprint title at the top) instead of manually prefixing each function. Namespaces make the origin blueprint of every function obvious wherever it is referenced. Document intent directly on the canvas: rename blocks via `Modify name` in the [block options](https://docs.rual.nl/blueprints/tips-and-tricks#block-options), and use the context menu's [comment, marker, and warning](https://docs.rual.nl/blueprints/introduction#context-menu) blocks to explain non-obvious flows and warn collaborators away from fragile areas. ## Using Functions for Reusability Custom functions are the unit of reuse in RUAL. Create them anywhere with the context menu's `Add function`, collect shared ones in a dedicated function blueprint under a namespace, and call them with [`function_custom_execute`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute). - **Private by default**: keep functions private until another blueprint needs them. Public functions are your API surface: changing them affects every caller. - **Pass minimal data**: hand a function a `guid` instead of an entire object. This keeps calls cheap and the contract explicit. - **Return through out-pins**: expose results as named out-pins and order them with [Edit pin sorting](https://docs.rual.nl/blueprints/tips-and-tricks#edit-pin-sorting) when a function grows several outputs. Do *not* extract a function for logic used exactly once, or when extraction would force you to pass half the flow's state as parameters. A well-placed [group with a title](https://docs.rual.nl/blueprints/introduction) around the blocks often documents the flow better than a premature function. > **Remember: function calls wait** The [`function_custom_execute`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute) block waits for the function to complete before the flow continues. If the work does not need to finish before the user gets a response, offload it to the [queue](https://docs.rual.nl/blueprints/queue) instead. ## Performance Optimization Most RUAL performance problems come from three sources: repeated storage queries, unbounded searches, and heavy work blocking user-facing flows. The fixes: | Technique | When to apply it | | --- | --- | | [Reuse blocks for identical values](https://docs.rual.nl/blueprints/block-execution#precompiled) | Always: precompiled data makes one shared block cheaper than duplicated ones. | | [Cache in Redis](https://docs.rual.nl/blueprints/storage#caching) | Search results that are requested far more often than the underlying data changes. Set a TTL and invalidate through [storage events](https://docs.rual.nl/blueprints/storage-events). | | Cache aggregated statistics in Redis | Counts and rollups: update them with [repeating events](https://docs.rual.nl/blueprints/repeating-events) instead of running aggregation queries on heavily populated storages. | | [Offload to the queue](https://docs.rual.nl/blueprints/queue) | Computationally intensive or deferrable work like PDF generation or sending email. Never keep a user waiting for it. | | Bound every search | Always set a limit, prefer [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) (no scoring), return only needed fields with [`query_source_by_field`](https://docs.rual.nl/block-types/query/query_source_by_field), and stream large sets with [`function_search_stream`](https://docs.rual.nl/block-types/storage/function_search_stream). | Choose cache keys that identify the exact query, including the parameters: so a cached entry is never served for a different request. And reach for `get document` by `_meta.guid` whenever you have one: it is the fastest retrieval method and the easiest to cache. The math that justifies caching: a single search is cheap, but the same query executed by 300 simultaneous page visits in one minute is not: checking the cache first and only hitting storage when necessary keeps the system fast at scale. Note that saving a blueprint discards its [precompiled data](https://docs.rual.nl/blueprints/block-execution#precompiled) on all production nodes, and a node clears precompiled data for blueprints it has not executed in 6 hours: so first requests after a save or an idle period are the expensive ones, and a cache absorbs exactly those. > **Real-time data is the exception** Searches do not wait for the latest inserts by default, and that is fine for most reads. Reserve the [`disable cache`](https://docs.rual.nl/block-types/query/query_disabled_request_cache) query option for flows that truly cannot tolerate stale results: on high-insert storages it can slow every search. See [Common Pitfalls](https://docs.rual.nl/blueprints/common-pitfalls). ## Managing Large Blueprints Some blueprints legitimately stay big. A complex page can have dozens of blocks. Keep them workable with the built-in organization tools: - **Groups**: draw groups with custom titles around blocks that form a logical step. Groups are the table of contents of your canvas. - **Block search**: `Ctrl + F` jumps straight to any block or value. The fastest way into an unfamiliar blueprint. - **Minimap**: keep the resizable minimap open to maintain orientation while you pan. - **Markers**: drop a marker from the context menu to flag a position for collaborators. - **Tags**: tag blueprints from the top bar so their purpose is visible in the overview. - **Explorer**: use the Explorer button to find all blocks of a type: for example every API block, in one list. - **Audit log**: every modification is logged automatically. Use a block's `Revisions` option to see the history of just that block when something changed unexpectedly. - **Copy, don't rebuild**: when a flow already exists in another blueprint, copy the blocks over instead of recreating them: then extract a shared function if the copy is the second one. If several of these tools feel necessary just to stay oriented, take it as the signal from [When to Split Blueprints](#split-blueprints): the canvas is telling you it holds more than one component. ## Team Habits Blueprints are a shared canvas with real-time collaboration, so team discipline matters as much as individual skill: - **Saving is not deploying**: since version 13.0.7, saved changes stay in development. Deploy deliberately through the `Deployment` menu, and use the `compare` button to review the difference between your development blueprint and what is live before you ship. See [How to Deploy](https://docs.rual.nl/deployment/how-to-deploy). - **Preview before you save**: append `?staging` to a page's URL to see your unsaved changes in action, then save with `Cmd/Ctrl + S` once you are satisfied. See [Staging and saving](https://docs.rual.nl/blueprints/tips-and-tricks#saving-blueprint). - **Review history, not memory**: the audit log records every change like git commits. Check a block's `Revisions` before "fixing" something a teammate may have changed on purpose. - **Communicate on the canvas**: warnings and comments travel with the blueprint. Use them instead of chat messages that disappear. - **Debug against real data safely**: use `Production Run` from the play mode options to execute the development blueprint against production data without touching the live environment: and remember this permission can be restricted per user in [User Access Management](https://docs.rual.nl/cluster/user-access-management#permissions). None of these habits cost time once they are default. Together they are what let multiple developers share a canvas without stepping on each other. - [Tips and Tricks](https://docs.rual.nl/blueprints/tips-and-tricks): The full productivity reference: searching, selecting, staging, namespaces, and the minimap. - [Block Execution](https://docs.rual.nl/blueprints/block-execution): Execution order, precompiled data, error handling, and async patterns. - [Common Pitfalls](https://docs.rual.nl/blueprints/common-pitfalls): Cache usage, real-time search, and aggregation misuse to avoid. ## Frequently asked **When should I split a RUAL blueprint into multiple blueprints?** Dedicate each blueprint to a single component: a page, a modal, an API endpoint or a set of related functions. Signs a blueprint has grown too big: you need the minimap and Ctrl + F for every navigation, it mixes unrelated responsibilities, or new team members cannot tell what it is from the overview. **How do I share logic between RUAL blueprints?** With public functions. Create a trigger_custom_function block in a dedicated function blueprint under a namespace and call it from anywhere else with function_custom_execute. Functions are private by default, so make them public only when another blueprint genuinely needs them. **How do I make RUAL storage searches faster?** Always set a limit, prefer query_bool_filter because it does no scoring, return only the fields you need with query_source_by_field, and stream large sets with function_search_stream. For results requested far more often than the data changes, cache in Redis and invalidate through storage events. **Does a RUAL function call block the rest of the flow?** Yes. function_custom_execute waits for the function to complete before the flow continues, so work that does not need to finish before the user gets a response belongs on the queue instead. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Common Blueprint Patterns · RUAL Documentation" description: "Reusable shapes for validation, mapping arrays, caching, error handling and API auth." canonical: https://docs.rual.nl/blueprints/common-patterns language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Common Blueprint Patterns Copyable blueprint recipes for everyday tasks: form validation, API auth, the JSON response backbone, data transformation, error handling, and caching with Redis and storage events. Each pattern below is a mini-recipe you can copy straight onto your canvas: the goal, when to use it, the blocks involved with their wiring, and notes from practice. Every block mentioned exists in the current block library. Search its type name in the blueprint editor to add it. ## 1. Form Validation **Goal:** Reject incomplete submissions before they reach storage. **When to use:** Any state page that accepts user input through a form and writes it to storage or passes it to a function. **Recipe:** | Block | Purpose | Connects To | | --- | --- | --- | | [`state_form`](https://docs.rual.nl/block-types/state%20ui/state_form) (form) | Renders the form and collects user input into the state. | Submit button triggers [`state_form_get`](https://docs.rual.nl/block-types/state%20ui/state_form_get) | | [`state_form_get`](https://docs.rual.nl/block-types/state%20ui/state_form_get) (get form data) | Retrieves the submitted form data from the state. Exposes a `form-object` out-pin with the field values and a `form-error` value pin for form-level errors. | Condition blocks, one per required field | | [`condition_not_empty_value`](https://docs.rual.nl/block-types/condition/condition_not_empty_value) (is not empty) | Returns `true` when the field value from `form-object` is filled. | A condition branch guarding the save flow | | [`function_console_log`](https://docs.rual.nl/block-types/logging/function_console_log) (log) | Optional: writes validation failures to the blueprint console while developing. | The failing branch of the condition | Wire every required field of the `form-object` into its own [`condition_not_empty_value`](https://docs.rual.nl/block-types/condition/condition_not_empty_value) block. Only when all conditions pass does the flow continue to the create or update document blocks; otherwise it branches to an error message in the UI. **Notes:** - Validate *before* touching storage. A rejected submission should never produce a document. - Keep one condition block per field instead of one combined check, so the UI can tell the user exactly which field failed. - Inspect the `form-error` pin first: form-level errors mean the submitted data could not be read at all. ## 2. API Authentication **Goal:** Expose an API endpoint only to the right callers, with explicit error replies for everyone else. **When to use:** Any custom API built with an `on_startup_register_uri_{method}` block that should not be fully public. **Recipe:** | Block | Purpose | | --- | --- | | [`on_startup_register_uri_get`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_get) (on api get) | Exposes a GET URI on `/api/`. Set its scopes through the lock icon: `*public` (default, everyone), `*loggedin` (authenticated users only), or a custom scope. | | [`user_current`](https://docs.rual.nl/block-types/users%2Cglobals/user_current) (current user) | Returns the logged-in user object, so the flow can identify who is calling. | | [`httpconnection_current_request`](https://docs.rual.nl/block-types/http%20connection/httpconnection_current_request) (current request) | Retrieves a reference to the current HTTP connection, needed to send an explicit reply. | | [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) (reply in json) | Replies to the connection with a JSON body and a status `code` in-pin. Use `401` for unauthenticated and `403` for unauthorized callers. | With the `*loggedin` scope, the cluster rejects requests without a valid `access_token` before your flow runs. Inside the flow, [`user_current`](https://docs.rual.nl/block-types/users%2Cglobals/user_current) plus a condition on the user's scopes lets you add finer-grained checks and answer failures yourself through [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json). **Notes:** - Callers can pass the token five ways: `Authorization: Bearer` header, `x-authtoken` or `x-token` header, an `access_token` cookie, or the `?access_token=` query parameter. See [Adding tokens to your requests](https://docs.rual.nl/cluster/api-guide#token-providing). Tokens are valid for 14 days and extend automatically while actively used ([Token Expiry](https://docs.rual.nl/cluster/api-guide#token-expiry)). - Protect the endpoint further with a [rate limit or throttle](https://docs.rual.nl/blueprints/remote-access-control) in a few clicks. - An endpoint that still returns 404 after scoping usually was never activated. Press `Activate` and deploy for production traffic. See [API not activating](https://docs.rual.nl/troubleshooting/common-issues#api-not-activating). ## 3. The JSON API Response Backbone **Goal:** Answer an API call with a clean, shaped JSON body. **When to use:** Every custom endpoint handler. This chain is the most common wiring in production blueprints by a wide margin: across the production blueprints on our own cluster, [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) is fed by [`httpconnection_current_request`](https://docs.rual.nl/block-types/http%20connection/httpconnection_current_request) more than any other pair (541 wires), and [`object_field_getter_multiple`](https://docs.rual.nl/block-types/object%2Cfields/object_field_getter_multiple) is the most-used non-input block of all (571 instances across 79 blueprints). **Recipe:** | Block | Role in the chain | | --- | --- | | [`on_startup_register_uri_get`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_get) (on api get) | Registers the endpoint; its function pin roots the handler function. | | [`httpconnection_current_request`](https://docs.rual.nl/block-types/http%20connection/httpconnection_current_request) (current request) | The connection reference: feeds the input readers and the reply block. | | [`httpconnection_get_body`](https://docs.rual.nl/block-types/http%20connection/httpconnection_get_body) (get body) / [`httpconnection_get_params`](https://docs.rual.nl/block-types/http%20connection/httpconnection_get_params) (get params) | Reads the request input: the body object on writes, the URL parameters on reads. | | [`object_field_getter_multiple`](https://docs.rual.nl/block-types/object%2Cfields/object_field_getter_multiple) (get fields) | Outputs one pin per selected field of the request or document object, so every following block wires a named field instead of digging into an object. | | [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields) (new object) | Builds the response object with only the fields the caller may see. | | [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) (reply in json) + [`number_default`](https://docs.rual.nl/block-types/number/number_default) (number) | Sends the object with an explicit status `code`: 200 on the happy path, 4xx on the guarded paths. | **Notes:** - Reply on every path, including validation failures and not-found branches. A caller left without a reply sees a timeout, not an error. - Shape the response with [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields) even when the document already looks right: it keeps `_meta` internals and future private fields from leaking into the API. - Reading input fields with [`object_field_getter_multiple`](https://docs.rual.nl/block-types/object%2Cfields/object_field_getter_multiple) instead of ad-hoc object access keeps the handler readable when the payload grows. ## 4. Data Transformation **Goal:** Turn storage documents into exactly the shape your API or UI needs: no more, no less. **When to use:** Shaping API responses, converting arrays of documents, and writing denormalized data. **Recipe A: shape a response object:** Feed the fields you want to expose into an [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields) (new object) block, which builds a fresh object containing only those fields. Use [`object_update_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_update_fields) when you want to adjust an existing object instead of building a new one. This keeps internal fields such as `_meta` internals out of your API responses. ``` { "guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "username": "joe", "email": "joe@example.com" } ``` **Recipe B: transform an array:** Connect the array to an [`array_map`](https://docs.rual.nl/block-types/array/array_map) (map) block. It applies the connected function to every element and returns a new array with the results: the way to run per-element logic (formatting, lookups, parsing) over a [`function_search`](https://docs.rual.nl/block-types/storage/function_search) result. It accepts a `concurrency` number in-pin and exposes an `error` value out-pin; [`array_map_parse`](https://docs.rual.nl/block-types/array/array_map_parse) (map and parse) is the variant for parsing each element along the way. For simply picking fields off each document, the getter-plus-object shape from the response backbone above is what production blueprints use most; reach for map when each element needs its own flow. **Recipe C: denormalized writes:** Because RUAL storage has no relationships, include all data you need inside each document. Build the fields with [`mutations_set_bp_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_set_bp_field_multiple) and write them with [`function_create_document_from_mutations`](https://docs.rual.nl/block-types/storage/function_create_document_from_mutations) (create document) or [`function_update_document_mutations`](https://docs.rual.nl/block-types/storage/function_update_document_mutations) (update document). When a source document changes, keep the embedded copies in sync with a [`storage_event`](https://docs.rual.nl/block-types/storage/storage_event) flow. See [Data Modelling Without Relations](https://docs.rual.nl/blueprints/storage#data-modelling). ## 5. Error Handling **Goal:** Make failures visible, answer them properly, and retry work that deserves a second attempt. **When to use:** Every flow that touches storage, HTTP requests, or any other operation that can fail. RUAL has no try/catch. Blocks that can fail expose a `success` condition pin and an `error` value pin, and the flow keeps running unless you branch on them. **Recipe: log and reply (error_demo style):** | Step | Wiring | | --- | --- | | 1. Branch on the result | Connect the block's `success` pin to a condition block: `true` continues the happy path, `false` enters the error path. | | 2. Log the failure | Connect the `error` pin to the `message` in-pin of [`function_console_log`](https://docs.rual.nl/block-types/logging/function_console_log) (log), so the failure appears in the blueprint console. | | 3. Reply with an error | Reply through [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) with a `4xx`/`5xx` code for APIs, or show an error message in the UI for pages. | A typical error payload looks like this: ``` { "success": false, "error": "Document not found for the given GUID" } ``` **Notes:** - **Retry via the queue:** for background work that may fail transiently, (re)schedule it with [`function_custom_execute_from_queue`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute_from_queue) (execute in queue). It runs the function at a given time without waiting, and accepts an optional `unique_id` and `debounce` to avoid duplicate runs. See [Queue](https://docs.rual.nl/blueprints/queue). - **Audit failures yourself:** the blueprint audit log records *modifications*, not runtime errors. For a failure trail, write errors to a dedicated storage with [`function_create_document_from_mutations`](https://docs.rual.nl/block-types/storage/function_create_document_from_mutations) or send them to an external webhook. The patterns from [Error Handling](https://docs.rual.nl/blueprints/block-execution#error-handling). - An unconnected `error` pin means failures pass silently. Always wire it somewhere. See [Reading error messages](https://docs.rual.nl/troubleshooting/debugging#error-messages). ## 6. Caching Strategies **Goal:** Stop paying for the same query twice. **When to use:** Read-heavy flows and high-traffic pages. A single search is cheap, but 300 simultaneous page visits run it 300 times a minute. **Recipe A: built-in search cache:** [`function_search`](https://docs.rual.nl/block-types/storage/function_search) has a `Cache Key` value in-pin. Pass a key that identifies the query (for example the searched user GUID) and repeated identical searches are served from cache instead of hitting storage again. **Recipe B, manual Redis check-aside:** | Block | Role in the pattern | | --- | --- | | [`value_redis_cache_get_key`](https://docs.rual.nl/block-types/redis/value_redis_cache_get_key) (get cache) | Reads the cached value for a key before you query storage. | | [`function_redis_cache_get_exists`](https://docs.rual.nl/block-types/redis/function_redis_cache_get_exists) (get cache key) | Flow-based variant: returns an `exists` condition plus the value, ideal for the cache-hit / cache-miss branch. | | [`function_redis_cache_set_key_json`](https://docs.rual.nl/block-types/redis/function_redis_cache_set_key_json) (set json) / [`function_redis_cache_set_key`](https://docs.rual.nl/block-types/redis/function_redis_cache_set_key) (set) | On a cache miss, run the search and store the result (JSON or string) under the key. Both accept an optional `ttl` in seconds. | | [`function_redis_cache_ttl`](https://docs.rual.nl/block-types/redis/function_redis_cache_ttl) (set cache ttl) | Adjusts the time-to-live of an existing key. | **Recipe C: invalidate on write:** cached data must die when the underlying documents change. Add a [`storage_event`](https://docs.rual.nl/block-types/storage/storage_event) (trigger event) flow on the storage. It fires on created, updated, removed, and saved events: and remove the affected keys with [`function_redis_cache_delete_key`](https://docs.rual.nl/block-types/redis/function_redis_cache_delete_key) (delete), or overwrite them with fresh data right away. **Recipe D: precompiled static lookups:** for values that never change, reuse one block for identical values across the blueprint instead of duplicating blocks. RUAL [precompiles](https://docs.rual.nl/blueprints/precompiled) block outputs for efficiency and reuses them on every execution. **Notes:** - Never cache what must be real-time. If a screen must show the absolute latest data, prefer the [`disable cache`](https://docs.rual.nl/block-types/query/query_disabled_request_cache) query option over cached results: and read the trade-off in [Common Pitfalls](https://docs.rual.nl/blueprints/common-pitfalls#disable-cache-query). - Invalidate narrowly: delete or overwrite the keys affected by the write instead of flushing broad key ranges. - Background on cache strategy lives in [Caching with Redis](https://docs.rual.nl/blueprints/storage#caching) and [Common Pitfalls](https://docs.rual.nl/blueprints/common-pitfalls). - [Block Execution](https://docs.rual.nl/blueprints/block-execution): Execution semantics behind these patterns: dataflow, parallel branches, and error pins. - [Common Pitfalls](https://docs.rual.nl/blueprints/common-pitfalls): The mistakes these patterns help you avoid: cache misuse, real-time search, and aggregations. ## Frequently asked **How do I validate a form before saving it in RUAL?** Read the submitted data with state_form_get and wire every required field of the form-object into its own condition_not_empty_value block. Only when all conditions pass does the flow continue to the create or update blocks, so a rejected submission never produces a document. **How do I reply with JSON from a RUAL API endpoint?** The standard chain is on_startup_register_uri_get into httpconnection_current_request, reading input with httpconnection_get_body or httpconnection_get_params, shaping the response with object_new_fields, and sending it with httpconnection_set_json plus an explicit status code. Reply on every path, including failures, or the caller sees a timeout instead of an error. **How do I cache a search result in RUAL?** Pass a Cache Key that identifies the exact query to function_search, or use the Redis blocks directly: check the key with function_redis_cache_get_exists, store the result with function_redis_cache_set_key_json and a ttl, and delete the key from a storage_event flow when the underlying documents change. **How do I handle errors in a RUAL flow?** RUAL has no try/catch: blocks that can fail expose a success condition pin and an error value pin, and the flow keeps running unless you branch on them. Connect success to a condition, log the error pin to function_console_log, and reply with a 4xx or 5xx code on the guarded paths. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Block Execution · RUAL Documentation" description: "The order blocks run in, how flow pins drive execution, and where values are resolved." canonical: https://docs.rual.nl/blueprints/block-execution language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Block Execution Understand how blocks execute within blueprint flows: dataflow between blocks, parallel execution of branches, error handling patterns, and debugging techniques. ## Dataflow Between Blocks In RUAL, data moves between blocks through pin connections. Each block can have input pins (in-pins) and output pins (out-pins). When you connect an out-pin of one block to an in-pin of another, you establish a data pathway. The system uses two distinct types of connections: - **Flow pins**: These control *when* a block executes. A block with a flow in-pin will only execute when it receives the flow signal from a preceding block. - **Data pins**: These carry typed data (value, number, condition, object, array, date, file, query, mutations, etc.) between blocks. Data pins define *what* information a block receives or produces. Blocks that do not have a flow in-pin: such as [`value_default`](https://docs.rual.nl/block-types/value/value_default), [`condition_is_true`](https://docs.rual.nl/block-types/condition/condition_is_true), or [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter): are evaluated when the block they connect to needs their data. This means data-only blocks are resolved on demand as part of the execution chain. ## Execution Order Execution follows the flow pin connections. When a flow starts: for example from a [`trigger_custom_function`](https://docs.rual.nl/block-types/globals%2Cfunction%20execution/trigger_custom_function) block. It sends a flow signal to the next connected block. That block executes, then passes the flow signal onward. When a block has multiple flow out-pins, or when one flow out-pin connects to multiple blocks, the resulting branches **execute in parallel**. This means RUAL does not wait for one branch to finish before starting another. ## Parallel Execution When a flow splits into multiple paths, all branches execute simultaneously. If you need to ensure a specific execution order, connect your blocks sequentially through a single flow path. For operations that must not run concurrently, consider using [`lock & wait`](https://docs.rual.nl/block-types/locking/function_claim_lock_wait) blocks. ## Precompiled Data To ensure blocks operate at peak efficiency, RUAL pre-builds the output values of blocks in advance. This optimized data is called [precompiled](https://docs.rual.nl/blueprints/precompiled) data. Each time a blueprint is saved, all production nodes discard their existing precompiled data. If a blueprint has not been executed on a node for at least 6 hours, the node automatically clears its precompiled information to conserve memory. For best performance, reuse the same block for identical values within a blueprint: more connections to a single block are more efficient than duplicating blocks with the same value. When calling functions, only pass the data you actually need (for example, just a `guid` instead of an entire object). ## Error Handling When a block encounters an error, the flow does **not** stop. Instead, most blocks that can fail provide specific output pins to communicate the result: - `success`. A condition pin that returns `true` if the operation succeeded and `false` if it failed. - `error`. A value pin that contains the error message when the operation fails. It is the developer's responsibility to check these pins and decide how to handle errors. Common patterns include: - Checking the `success` pin and showing an error message in the UI. - Logging errors to storage or sending them to an external webhook (for example, an email notification). - Using a condition block to branch the flow based on whether the operation succeeded. ``` { "success": false, "error": "Document not found for the given GUID" } ``` > **No try/catch equivalent** Unlike traditional programming, RUAL does not have a try/catch mechanism. Each block handles its own errors and exposes the result through its output pins. Always check the `success` and `error` pins on blocks that perform storage operations, HTTP requests, or other operations that can fail. ## Debugging Blueprint Execution RUAL provides several tools for debugging your blueprint flows: #### Simulation Mode Certain blocks, particularly those at the beginning of a flow, feature a play icon. Clicking this icon opens simulation options where you can set custom input values and execute the flow in simulation mode. The simulation traces through the connected blocks, showing you the output at each step. This is particularly useful for end-stage debugging. - [Simulation Guide](https://docs.rual.nl/blueprints/tips-and-tricks#simulation-functions): Learn how to configure simulation values for different pin types. #### Production Run When you select `Production Run` within the play mode options, it executes the flow using the non-deployed (development) blueprint but against production data. This allows you to debug with real data without affecting the live production environment. Note that the `Production Run` permission can be disabled for specific users in [User Access Management](https://docs.rual.nl/cluster/user-access-management#permissions). #### Console The blueprint console can be activated through the `Options` menu in the top right of your blueprint. The console shows execution details and can help identify issues in your flows. #### Audit Log Every blueprint modification is automatically logged. You can view the audit log through the `Options` menu or by clicking on a specific block's options () and selecting `Revisions` to see changes affecting only that block. ## Asynchronous Patterns While flows execute their blocks based on flow pin connections, there are several ways to offload work to run asynchronously: | Pattern | Use Case | | --- | --- | | [Queue](https://docs.rual.nl/blueprints/queue) | Offload computationally intensive tasks (like PDF generation) to background processing. The queue executes a function at a specified time, potentially on another node. Use the [`function_custom_execute_from_queue`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute_from_queue) block. | | [Storage Events](https://docs.rual.nl/blueprints/storage-events) | React to document changes (create, update, remove) in a specific storage. These events execute asynchronously after the document operation completes, on a FIFO basis. Use the [`storage_event`](https://docs.rual.nl/block-types/storage/storage_event) block. | | [Repeating Events](https://docs.rual.nl/blueprints/repeating-events) | Schedule recurring tasks that execute at a specified interval. Similar to `setInterval()` or `crontab`. Use the [`schedule_repeating_event`](https://docs.rual.nl/block-types/events/schedule_repeating_event) block. | > **Important** The [`function_custom_execute`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute) block executes a function and **waits** for it to complete before continuing the flow. If you want to run something in the background without waiting, use the [`queue`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute_from_queue) instead. ## Locking and Concurrency When multiple flows could modify the same data simultaneously, you can use locking blocks to prevent race conditions: - [`function_claim_lock_wait`](https://docs.rual.nl/block-types/locking/function_claim_lock_wait): Sets a lock or waits until an existing lock has been released. - [`function_claim_lock`](https://docs.rual.nl/block-types/locking/function_claim_lock_v2): Tries to claim a lock and returns whether it was successful. - [`function_claim_unlock`](https://docs.rual.nl/block-types/locking/function_claim_unlock): Releases an existing lock. Note that while the storage system itself is transaction-based (updates are processed sequentially per document), locking is useful when you need to ensure a broader set of operations runs atomically. ## Best Practices - **Keep flows focused**: Each flow should have a single responsibility. Use functions to break complex logic into smaller, reusable pieces. - **Use namespaces**: When managing many functions across blueprints, use the [namespace feature](https://docs.rual.nl/blueprints/tips-and-tricks#blueprint-namespace) to organize them clearly. - **Check error pins**: Always handle the `success` and `error` pins on blocks that interact with storage, external APIs, or other operations that can fail. - **Offload heavy work**: Use the [queue](https://docs.rual.nl/blueprints/queue) for tasks like PDF generation, email sending, or batch processing that don't need to complete immediately. - **Use caching**: Leverage Redis cache to avoid repeated storage queries. See [Common Pitfalls](https://docs.rual.nl/blueprints/common-pitfalls) for guidance on proper cache usage. ## Frequently asked **In what order do blocks execute in a RUAL blueprint?** Execution follows the flow pin connections: a block runs when it receives the flow signal, then passes it onward. When a flow splits into multiple paths, all branches execute in parallel, so connect blocks sequentially through a single flow path when a specific order matters. **When are data-only blocks evaluated in RUAL?** Blocks without a flow in-pin, such as value_default, condition_is_true or query_bool_filter, are evaluated on demand when the block they connect to needs their data, resolved as part of the execution chain. **Does RUAL have a try/catch mechanism?** No. Each block handles its own errors and exposes the result through a success condition pin and an error value pin, and the flow does not stop on failure. It is the developer's responsibility to check those pins on every block that can fail. **What is precompiled data in RUAL?** RUAL pre-builds the output values of blocks in advance for efficiency. Every save of a blueprint discards the precompiled data on all production nodes, and a node clears it for blueprints it has not executed in 6 hours, so the first requests after a save or an idle period are the expensive ones. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Object References and Copies · RUAL Documentation" description: "Objects are passed by reference: which blocks change the original, and when to take a shallow or deep copy." canonical: https://docs.rual.nl/blueprints/object-references language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Object References and Copies Objects on the canvas are passed by reference. Updating one can change what another block already read. This page shows when that happens and how to take a copy. An object pin does not carry a snapshot of the object. It carries the object itself. Two blocks connected to the same object out-pin are looking at the same thing in memory, so a change made through one of them is visible to the other, including to blocks that already ran. That is what makes flows cheap: a document of a few hundred fields is passed around without being copied on every hop. It is also the source of a specific class of bug, where a value changes underneath a block that looked at it earlier. ## Which blocks change the original | Block | What it does with the input | | --- | --- | | [`object_update_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_update_fields) | Applies the mutations and field pins **to the object you connected**, then outputs that same object. The original changes. | | [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields) | Builds a fresh object from its in-pins. Nothing you connected is modified. | | [`object_shallow_copy`](https://docs.rual.nl/block-types/object%2Cfields/object_shallow_copy) | Builds a new object with the same top-level keys. Nested objects and arrays are still shared with the original. | | [`object_deep_copy`](https://docs.rual.nl/block-types/object%2Cfields/object_deep_copy) | Builds a new object with no shared references at any depth. The safest, and the most expensive. | > **The rule of thumb** Reading is free. The moment a flow **writes** to an object it did not create, decide first whether the writer should own a copy. If any other branch of the flow still needs the untouched values, copy before you write. ## Shallow copy versus deep copy A shallow copy protects the top level only. Take a customer object with a nested `address`: after a shallow copy, setting `status` on the copy leaves the original alone, but setting `address.city` changes the original as well, because both objects point at the same nested address. A deep copy rebuilds every level, so nothing is shared. It costs time and memory proportional to the size of the object, which is why the block's own description warns against widespread use. Reach for it when you genuinely need an independent snapshot, for example when you keep a "before" version of a document to compare against or to log. ``` original = { "name": "joe doe", "address": { "city": "Amsterdam" } } copy = shallow_copy(original) copy.name = "sami b" // original.name is still "joe doe" copy.address.city = "Utrecht" // original.address.city is now "Utrecht" as well copy = deep_copy(original) copy.address.city = "Utrecht" // original.address.city stays "Amsterdam" ``` ## Where this bites in practice - **Keeping a before and after.** Reading a document, updating it in place, and then trying to log "what it was" gives you the new values twice. Deep-copy the document before the update if you want the old state. - **Iterations.** Updating the object of the current item inside a loop mutates the entry in the array you are iterating over. That is fine when it is what you meant, and confusing when it is not. - **Reusing a template object.** Building one base object and updating it per iteration leaves every reference pointing at the final state. Build a new object per iteration, or copy the template inside the loop. - **Passing to a function.** A custom function that updates the object it received changes the caller's object too. Copy at the boundary if the function is meant to be a pure transformation. ## Storage documents follow the same rule A document read from storage is an ordinary object on the canvas, so everything above applies to it. Writing to it does not write to storage: persisting still goes through mutations and [`function_update_document`](https://docs.rual.nl/block-types/storage/function_update_document_mutations), described in [Storages](https://docs.rual.nl/blueprints/storage). What in-place updates do change is what the rest of your flow sees. ## Next steps - [Locking and Concurrency](https://docs.rual.nl/blueprints/locking): The other half of the story: what happens when two flows touch the same data at the same time. - [Block Execution](https://docs.rual.nl/blueprints/block-execution): The order blocks run in, and when a value is resolved. - [Common Pitfalls](https://docs.rual.nl/blueprints/common-pitfalls): The other mistakes that cost an afternoon. ## Frequently asked **Does updating an object in RUAL change the original?** It depends on the block. object_update_fields applies its changes to the object you connected and outputs that same object, so the original changes. object_new_fields builds a new object and leaves the input untouched. **What is the difference between a shallow copy and a deep copy in RUAL?** A shallow copy creates a new object with the same top-level keys, but nested objects and arrays are still shared with the original. A deep copy rebuilds every level, so nothing is shared. Deep copies cost time and memory proportional to the object size. **How do I keep the old version of a document before updating it in RUAL?** Take a deep copy before you apply the update. Updating in place changes the object every other block is holding, so without a copy the before and after values are the same. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Locking and Concurrency · RUAL Documentation" description: "Document updates lock themselves, file actions do not. How and when to claim a lock yourself." canonical: https://docs.rual.nl/blueprints/locking language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Locking and Concurrency Document updates take a lock for you. File actions do not. This page shows which operations are safe on their own, and how to claim a lock yourself for the ones that are not. Your blueprints run on more than one node, and the same flow can be running several times at once: two users clicking at the same moment, a queue worker next to a page request, a repeating event that overlaps its previous run. Whether that is safe depends entirely on which operation you are performing. ## Document actions lock themselves Updating a storage document is guarded for you. The update takes a lock on that document, applies the mutations, and releases it, so two concurrent updates queue up instead of overwriting each other. This is what makes counters and array mutations safe: an increment is applied to the value as it is at that moment, not to the value your flow read a second ago. On top of that, a document carries an `update_hash` in its `_meta`. An update that supplies the hash it read is rejected with a conflict when another writer got there first, which is the check to use when you want "only write if nothing changed since I read it" rather than "apply my change on top". > **Read, then decide, then write** The lock covers the write, not your thinking. If your flow reads a document, decides something, and writes the result, another flow can still slip in between the read and the write. Express the decision as a mutation (increment, add to array) so it is applied atomically, or take a lock around the whole sequence. ## File actions do not lock File actions have no equivalent guard. Appending to a file, writing one, copying or deleting one: each simply performs the operation. Two flows appending to the same file at the same time interleave their writes, and a flow that reads a file, changes the contents and writes it back loses whatever the other flow wrote in between. Anything that touches a shared file therefore needs an explicit lock. This is the single most common cause of "the file is missing lines" and "the counter file went backwards". ## Claiming a lock | Block | Behaviour | | --- | --- | | [`function_claim_lock_v2`](https://docs.rual.nl/block-types/locking/function_claim_lock_v2) | Tries to claim the lock once and reports on its `success` out-pin whether it got it. Use it when the flow can do something sensible on failure, such as replying "busy, try again". | | [`function_claim_lock_wait`](https://docs.rual.nl/block-types/locking/function_claim_lock_wait) | Waits until the lock is free, retrying every two seconds, then continues. Use it when the work must happen and the caller can wait. | | [`function_claim_unlock`](https://docs.rual.nl/block-types/locking/function_claim_unlock) | Releases the lock for a key. | Both claim blocks take a `key` and an `expiry (sec.)`, which defaults to `60`. The expiry is a safety net: if a run dies without releasing, the lock disappears on its own instead of blocking the next run forever. Pick a key that identifies the resource you are protecting, for example the file path or the document guid, and pick an expiry longer than the slowest realistic run. Since core 15.0.0 both claim blocks also have an `auto clear` pin. With it set, the lock is released when the flow ends, including when the flow fails, so you do not need a separate free lock block on every exit path. > **Always plan the release** A lock without a release is an outage waiting for its expiry to run out. Use `auto clear`, or make sure every path out of the flow, including the error path, reaches `function_claim_unlock`. ## The pattern for a shared file - Claim a lock keyed on the file path. - Read the file, change it, write it back. - Release the lock, or let `auto clear` do it. Keep the locked section as small as possible. Fetching an external API or resizing an image inside the lock makes every other run wait for it, and pushes you towards an expiry long enough to be dangerous. ## When you need a lock | Situation | Lock needed | | --- | --- | | Increment a counter on a document | No. Use a mutation, which is applied sequentially per document. | | Add a value to an array field on a document | No. The array mutations are applied the same way. | | Read a document, compute a new value from it, write it back | Yes, or supply the `update_hash` you read and handle the conflict. | | Append to a shared file, or rewrite a file in place | Yes, always. | | A repeating event whose run can outlast its interval | Yes. Claim a lock on the job name so runs cannot overlap. | | Calling an external API that must not be called twice for the same record | Yes, keyed on the record, in combination with a marker field so a retry is idempotent. | ## Next steps - [Object References and Copies](https://docs.rual.nl/blueprints/object-references): The in-memory half of the same problem: objects are shared until you copy them. - [Storages](https://docs.rual.nl/blueprints/storage): Mutations, transactional updates and the _meta fields, including update_hash. - [Repeating Events](https://docs.rual.nl/blueprints/repeating-events): Scheduling work, and keeping overlapping runs from colliding. ## Frequently asked **Do I need a lock to update a document in RUAL?** Usually not. A document update takes a lock on that document automatically, so concurrent updates queue instead of overwriting each other. You do need one when your flow reads a document, decides something, and writes the result, unless you supply the update_hash you read and handle the conflict. **Are file actions in RUAL safe when two flows run at the same time?** No. File actions have no automatic locking, unlike document actions. Two flows appending to the same file interleave their writes. Claim a lock keyed on the file path around any read, change and write back sequence. **How do I claim a lock in a RUAL blueprint?** Use function_claim_lock_v2 to try once and branch on its success out-pin, or function_claim_lock_wait to wait until the lock is free. Both take a key and an expiry in seconds that defaults to 60. Release with function_claim_unlock, or set the auto clear pin so the lock is released when the flow ends. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Storages · RUAL Documentation" description: "The JSON document model, _meta, transactional mutations and how to query documents." canonical: https://docs.rual.nl/blueprints/storage language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Storage RUAL includes a built-in, JSON-based storage system with full-text search. Learn how to create, update, search, and manage documents. ## Storing Data Blueprints include enterprise-scale storage, accessible through blocks like `search` and `create document`. The storage is JSON-based and schema-less. You don't need to predefine any data structure. The tradeoff is that there are no relationships between storage entities, so you should include all necessary data within each document (denormalization). ## Meta Data Every document includes a `_meta` key with the following fields: | Key | Description | | --- | --- | | `guid` | The unique identifier of the document. This is a generated hash and is immutable. | | `expiry` | A Unix timestamp for the [expiration date](#document-expiry). Defaults to `-1` (no expiry). | | `entity` | Identifies which entity within the cluster this document belongs to. | | `removed` | A Unix timestamp indicating removal. `0` means the document is active. | | `created` | Unix timestamp of when the document was created. | | `updated` | Unix timestamp of the last update. | | `ums` | Creation timestamp in milliseconds. | | `cms` | Last update timestamp in milliseconds. | | `update_hash` | A hash for the most recent modification, part of the [transaction-based](#updating-documents) update system. | ``` { "username": "Joe", "_meta": { "guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "expiry": -1, "entity": 1, "removed": 0, "created": 1706396400, "updated": 1706396400, "ums": 1706396400803, "cms": 1706396400362, "update_hash": "60e27209fa93063b5605e41605c2722ed428cae0" } } ``` ## Creating Documents To create a document, you need a `storage` reference and a set of `mutations` that define the fields: - Add a [`storage`](https://docs.rual.nl/block-types/globals%2Cstorage/storage) block and select the storage you want to write to. - Add a [`mutations_set_bp_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_set_bp_field_multiple) block to define the fields and their values. - Add a [`function_create_document`](https://docs.rual.nl/block-types/storage/function_create_document_from_mutations) block, connect the storage and mutations to its in-pins, and connect the flow pin from your flow. The `function_create_document` block returns the created document including the generated `_meta` with its unique `guid`. ``` { "username": "joe", "email": "joe@example.com", "role": "editor", "_meta": { "guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "expiry": -1, "entity": 1, "removed": 0, "created": 1706396400, "updated": 1706396400, "ums": 1706396400803, "cms": 1706396400362, "update_hash": "60e27209fa93063b5605e41605c2722ed428cae0" } } ``` ## Field Name Rules Field names are always stored in **lowercase**. If you use `UserName`, it will be stored as `username`. Fields cannot include dots (`.`). If they do, RUAL automatically converts them into a nested object structure. ## Updating Documents Storage uses a transaction-based update system through [mutations](https://docs.rual.nl/block-types/mutations). Each update is processed sequentially, so concurrent updates won't overwrite each other. This makes counters and array operations safe to use. To update a document, use the [`function_update_document`](https://docs.rual.nl/block-types/storage/function_update_document_mutations) block with the document's `guid` and the mutations you want to apply. Common mutation operations: | Block | Operation | | --- | --- | | [`mutations_set_bp_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_set_bp_field_multiple) | Set one or more fields to specific values. | | [`mutations_set_custom_field`](https://docs.rual.nl/block-types/mutations/mutations_set_custom_field) | Set a field using a dynamic key and value. | | [`mutations_increment_by_field`](https://docs.rual.nl/block-types/mutations/mutations_increment_by_field) | Increment a numeric field by a given amount. | | [`mutations_decrement_by_field`](https://docs.rual.nl/block-types/mutations/mutations_decrement_by_field) | Decrement a numeric field by a given amount. | | [`mutations_add_to_array_by_field`](https://docs.rual.nl/block-types/mutations/mutations_add_to_array_by_field) | Add a unique value to an array field. | | [`mutations_remove_from_array_by_field`](https://docs.rual.nl/block-types/mutations/mutations_remove_from_array_by_field) | Remove a value from an array field. | | [`mutations_remove_by_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_remove_by_field_multiple) | Remove one or more fields from the document entirely. | | [`mutations_add`](https://docs.rual.nl/block-types/mutations/mutations_add) | Combine multiple mutation sets into one. | The execution result includes additional keys since it operates on an existing document: ``` { "version": 130005, "title": "v13.0.5", "counter": 2, "_meta": { "guid": "723575f49d0af385314d1f02280163374297e018a692e5e4ab85eb907ebf6eb5", "expiry": -1, "entity": 1, "removed": 0, "created": 1706396400, "updated": 1708393100, "ums": 1706396400803, "cms": 1708393100132, "update_hash": "30e27209fa93073b5605e41605c2722ed448cae1" } } ``` ## Getting a Document by GUID The fastest way to retrieve a document is by its `_meta.guid` using the [`function_get_document`](https://docs.rual.nl/block-types/storage/function_get_document) block. This is faster than searching and easier to cache. Avoid creating custom unique identifiers like `code`. Use `_meta.guid` instead. To retrieve multiple documents at once, use the [`function_get_documents`](https://docs.rual.nl/block-types/storage/function_get_documents) block with an array of GUIDs. ## Building Search Queries Search queries are built by connecting query blocks together and feeding them into a [`function_search`](https://docs.rual.nl/block-types/storage/function_search) block. A typical search flow looks like this: - Create the query conditions using blocks like [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) (exact match on a field). - Wrap them in a boolean query using [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter), [`query_bool_must`](https://docs.rual.nl/block-types/query/query_bool_must), or [`query_bool_should`](https://docs.rual.nl/block-types/query/query_bool_should). - Optionally add sorting with [`query_sort_field`](https://docs.rual.nl/block-types/query/query_sort_field) and a limit with [`query_limit`](https://docs.rual.nl/block-types/query/query_limit) or a `number` block connected to the limit pin. - Combine all query parts using [`query_and`](https://docs.rual.nl/block-types/query/query_and). - Connect the final query output to the query in-pin of [`function_search`](https://docs.rual.nl/block-types/storage/function_search). - Select the storage on the `function_search` block. #### Search Query Example This example finds documents in a `messages` storage where `user_guid` matches a specific value, sorted by creation date descending, limited to 10 results: | Block | Purpose | Connects To | | --- | --- | --- | | `value_default` | The user GUID to search for | [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) (user_guid field) | | [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) | Match documents where `user_guid` equals the value | [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) | | [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) | Wrap the term query in a filter context | `query_and` | | [`query_sort_field`](https://docs.rual.nl/block-types/query/query_sort_field) | Sort by `_meta.created` descending | `query_and` | | `query_and` | Combine filter and sort into one query | `function_search` (query pin) | | `number_default` (10) | Limit results to 10 | `function_search` (limit pin) | The `function_search` block returns an array of matching documents. For queries that should return at most one result, use [`function_search_single_result`](https://docs.rual.nl/block-types/storage/function_search_single_result) instead. ## Query Types Overview | Query Block | What It Does | | --- | --- | | [`query_bool_must`](https://docs.rual.nl/block-types/query/query_bool_must) | All provided queries **must** match (AND logic). | | [`query_bool_should`](https://docs.rual.nl/block-types/query/query_bool_should) | One or more queries **should** match (OR logic). | | [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) | Filters results using a boolean query (does not affect scoring). | | [`query_bool_must_not`](https://docs.rual.nl/block-types/query/query_bool_must_not) | All provided queries **must not** match (exclusion). | | [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) | Exact match on a selected field. | | [`query_bool_range_field`](https://docs.rual.nl/block-types/query/query_bool_range_field) | Range comparison (greater than, less than) on a numeric field. | | [`query_bool_range_date_field`](https://docs.rual.nl/block-types/query/query_bool_range_date_field) | Range comparison on a date field. | | [`query_bool_exists_bpfield`](https://docs.rual.nl/block-types/query/query_bool_exists_bpfield) | Check if a field exists in the document. | | [`query_bool_wildcard_string_field`](https://docs.rual.nl/block-types/query/query_bool_wildcard_string_field) | Wildcard matching on a field value. | | [`query_bool_prefix_string_field`](https://docs.rual.nl/block-types/query/query_bool_prefix_string_field) | Check if a field value starts with a given prefix. | | [`query_bool_term_fuzzy`](https://docs.rual.nl/block-types/query/query_bool_term_fuzzy) | Find documents with terms similar to the search term (fuzzy matching). | ## Search Tips > **Case-Sensitive Searches** Search queries in RUAL storage are **case-sensitive**. If you store a value as `"Joe"` and search for `"joe"`, the document will not be found. It is good practice to store searchable values in lowercase to ensure accurate search results. - **Use `get document` when possible**: Retrieving by `_meta.guid` is the fastest method and easy to cache. Avoid custom unique identifiers. - **Use [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) for most queries**: Filter context is more efficient than `must` because it skips relevance scoring. - **Limit your results**: Always set a reasonable limit. Smaller result sets are faster to return and process. - **Select specific fields**: Use [`query_source_by_field`](https://docs.rual.nl/block-types/query/query_source_by_field) to return only the fields you need. - **Stream large result sets**: Use [`function_search_stream`](https://docs.rual.nl/block-types/storage/function_search_stream) to process results as they arrive rather than loading everything into memory. ## Real-Time Search Results By default, search queries do not wait for the most recent inserts to be indexed. If you create a document and immediately search for it, it might not appear yet. In most cases this slight delay is acceptable. If you need the absolute latest data, use the [`disable cache`](https://docs.rual.nl/block-types/query/query_disabled_request_cache) query block. This forces the system to wait until all pending inserts are indexed before running the search. In high-insert environments, this can introduce noticeable delays. - [Learn More](https://docs.rual.nl/blueprints/common-pitfalls#disable-cache-query): Read about real-time search results and when to use the disable cache option. ## Full-Text Search The storage includes a built-in full-text search engine. The main blocks for text searching are: - [`query_bool_simple_query_string_field`](https://docs.rual.nl/block-types/query/query_bool_simple_query_string_field): for user-facing search inputs. Handles common search syntax. - [`query_bool_wildcard_string_field`](https://docs.rual.nl/block-types/query/query_bool_wildcard_string_field): for pattern matching with wildcards. - [`query_bool_prefix_string_field`](https://docs.rual.nl/block-types/query/query_bool_prefix_string_field): for autocomplete-style "starts with" searches. - [`query_bool_term_fuzzy`](https://docs.rual.nl/block-types/query/query_bool_term_fuzzy): for fuzzy matching that tolerates typos. Remember that searches are case-sensitive. Store searchable text in lowercase and convert user input to lowercase before searching. ## Data Modelling Without Relations RUAL storage does not support relationships between storage entities. There are no SQL-style joins. Instead, denormalize your data: include all the information you need within each document. #### Example: Orders and Customers In a relational database, you'd have a separate `customers` table and reference the customer ID from `orders`. In RUAL, include the relevant customer information directly in the order document: ``` { "order_number": "ORD-2024-0042", "status": "shipped", "total": 15990, "customer_name": "joe doe", "customer_email": "joe@example.com", "customer_guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "items": [ { "name": "widget", "quantity": 3, "price": 5330 } ], "_meta": { "guid": "d56808172a3d26b86472153355496fa752ae341bc915e80b19de63a920e89203", "expiry": -1, "entity": 1, "removed": 0, "created": 1706396400, "updated": 1708393100, "ums": 1708393100132, "cms": 1706396400362, "update_hash": "3d82f70b5ce1946a208bd4f6019e7c5a86bb42d1" } } ``` This way, searching for orders gives you the customer name and email without a second query. If a customer changes their name, update all their order documents using a [storage event](https://docs.rual.nl/blueprints/storage-events) on the customers storage. #### Denormalization Strategies - **Embed frequently accessed data**: Store `customer_name` on every order instead of just `customer_guid`. - **Keep a reference GUID**: Always store the `guid` of related documents so you can `get document` when you need the full data. - **Use storage events for sync**: When a source document changes (e.g., customer name), use a [storage event](https://docs.rual.nl/blueprints/storage-events) to update all documents that embed that data. - **Store counters separately**: Instead of [aggregation queries](https://docs.rual.nl/blueprints/common-pitfalls#misuse-of-aggregations), maintain a counter field on a parent document and increment it when child documents are created. ## Document Lifecycle | Operation | Block | Notes | | --- | --- | --- | | Create | [`function_create_document`](https://docs.rual.nl/block-types/storage/function_create_document_from_mutations) | Returns the created document with `_meta`. Triggers `on_created` and `on_saved` storage events. | | Read | [`function_get_document`](https://docs.rual.nl/block-types/storage/function_get_document) | Fastest retrieval method by `guid`. Preferred over search for single documents. | | Search | [`function_search`](https://docs.rual.nl/block-types/storage/function_search) | Full-text search with query blocks. Results may have a slight indexing delay. | | Update | [`function_update_document`](https://docs.rual.nl/block-types/storage/function_update_document_mutations) | Transaction-based. Creates a revision if enabled. Triggers `on_updated` and `on_saved` events. | | Remove (soft) | [`function_remove_document`](https://docs.rual.nl/block-types/storage/function_remove_document) | Marks as removed but data stays. Excluded from search by default. Can be restored. | | Restore | [`function_restore_document`](https://docs.rual.nl/block-types/storage/function_restore_document) | Restores a removed document back to active state. | | Delete (permanent) | [`function_delete_document`](https://docs.rual.nl/block-types/storage/function_delete_document) | Permanently deletes. Cannot be recovered unless revisions are enabled. | > **Best Practice** Use `remove` (soft delete) in most cases instead of permanent `delete`. This allows recovery if needed. You can set up a [repeating event](https://docs.rual.nl/blueprints/repeating-events) to permanently purge removed documents after a retention period. ## Document Removal Documents can be removed and restored at any time. Use [`function_remove_document`](https://docs.rual.nl/block-types/storage/function_remove_document) or [`function_remove_multiple_documents`](https://docs.rual.nl/block-types/storage/function_remove_documents) to soft-remove documents. This is different from permanent deletion with [`function_delete_document`](https://docs.rual.nl/block-types/storage/function_delete_document). Removed documents are tagged as such, and a revision is created if revisions are enabled. Search queries exclude removed documents by default, but you can include them with the [`query_include_removed`](https://docs.rual.nl/block-types/query/query_include_removed) block. Restoring is straightforward with the [`function_restore_document`](https://docs.rual.nl/block-types/storage/function_restore_document) block. The data is never actually deleted from storage. A good practice is to remove documents first, then permanently purge them after a retention period. ## Document Expiry By default, documents don't expire (`_meta.expiry` is `-1`). You can set an expiry timestamp when creating or updating a document using the [`function_create_document`](https://docs.rual.nl/block-types/storage/function_create_document_from_mutations) and [`function_update_document`](https://docs.rual.nl/block-types/storage/function_update_document_mutations) blocks. To reset a document's expiry back to "never", use the [`date_never`](https://docs.rual.nl/block-types/date/date_never) block, which sets `_meta.expiry` to `-1`. There is a slight delay before an expired document is physically deleted, but the [`get document`](https://docs.rual.nl/block-types/storage/function_get_document) block treats it as deleted once the expiry time has passed. > **Cautious** A document set to expire will be **permanently** removed from the storage and **cannot** be recovered, unless revisions for that storage are activated. By default, revisions are not enabled. Restoring a deleted document is a manual task. ## Caching with Redis For high-traffic applications, cache search results in Redis instead of querying storage on every request. Common patterns: - Cache frequently accessed search results with a TTL using `redis cache` blocks. - Invalidate or update the cache when documents change, using [storage events](https://docs.rual.nl/blueprints/storage-events). - Store aggregated statistics in Redis and update them periodically with [repeating events](https://docs.rual.nl/blueprints/repeating-events). - [Common Pitfalls](https://docs.rual.nl/blueprints/common-pitfalls): Learn about proper cache usage, real-time search, and avoiding aggregation misuse. - [Protected Storages](https://docs.rual.nl/blueprints/protected-storages): Keep a storage usable by a flow and by your users while its contents stay out of the studio. - [Scaling and Automatic Sizing](https://docs.rual.nl/architecture/scaling): How a storage is sized from what it holds, and the few design choices on your side that matter. ## Frequently asked **What is _meta in a RUAL storage document?** _meta is the object every document carries with its guid, expiry, entity, removed state, created and updated timestamps, the ums and cms millisecond timestamps, and the update_hash of the last modification. It is never empty. **How do concurrent updates work in RUAL storage?** Updates go through mutations and are processed sequentially per document, so concurrent updates do not overwrite each other. That makes counters and array operations safe to use. **Can I use dots in a RUAL storage field name?** No. Field names are stored in lowercase and cannot contain dots. If a field name contains a dot, RUAL converts it into a nested object structure instead. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Storage Events · RUAL Documentation" description: "React to document creates and updates, including how rapid updates are combined." canonical: https://docs.rual.nl/blueprints/storage-events language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Storage Events When dealing with storage operations, it's important to note that document changes can impact the entire application. That's why we provide the ability to manage storage events, enabling you to trigger events when documents are created, updated, or removed within your storage system. ## Supported Event types At present, we offer support for the following storage events: | Event | Description | | --- | --- | | `on_created` | This event is triggered when a new document is created within the specified storage. | | `on_saved` | This event is triggered when a new document is created or an existing document is updated within the specified storage. | | `on_updated` | This event is triggered when an existing document is updated within the specified storage. | | `on_removed` | This event is triggered when an existing document is marked as removed within the specified storage. | ## Event Execution Storage Events operate on a First In, First Out (FIFO) basis, although simultaneous execution by multiple nodes is possible. For instance, while Node A is processing Event 61, Node B might already be executing Event 62, due to Node A concurrently processing Event 60, which Node B is not. Given the variability in processing speeds across different nodes, the FIFO execution logic for these events cannot be guaranteed. Within the blueprint function, you can manage the execution order using [lock & wait](https://docs.rual.nl/block-types/locking/function_claim_lock_wait) blocks. However, this approach may decelerate other events that are executing in the same thread and block. To ensure a 100% guarantee of execution order, it is advisable to offload the intensive tasks of Storage Events and transfer them to a [queue](https://docs.rual.nl/blueprints/queue). ## Event Payload When a document is created or updated, we will furnish you with the most recent document data, along with a list of fields that have undergone modifications. The document you receive represents the current state of the document at the moment of the event. Given the extremely rapid and high-performance nature of these events, they may occasionally be combined. To provide you with a clearer understanding of what might occur in such rare scenarios, we've included a JSON example below illustrating the functioning of Storage Events. ## Document Created Flow ``` { "username": "Joe", "_meta": { "guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "expiry": -1, "entity": 1, "removed": 0, "created": 1706396400, "updated": 1706396400, "ums": 1706396400803, "cms": 1706396400362, "update_hash": "60e27209fa93063b5605e41605c2722ed428cae0" } } ``` ``` { "object": { "username": "Joe", "_meta": { "guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "expiry": -1, "entity": 1, "removed": 0, "created": 1706396400, "updated": 1706396400, "ums": 1706396400803, "cms": 1706396400362, "update_hash": "60e27209fa93063b5605e41605c2722ed428cae0" } }, "revisions": [ { "key": "username", "value": "Joe", "previous_value": null } ], "fields_modified": [ "username" ] } ``` ## Document Updated Flow ``` { "username": "Joe Do", "firstname": "Joe", "lastname": "Do", "_meta": { "guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "expiry": -1, "entity": 1, "removed": 0, "created": 1706396400, "updated": 1706482800, "ums": 1706482800417, "cms": 1706396400362, "update_hash": "8fc1d0b6a4e37225b1904c7de0f5a8c6431bb27e" } } ``` ``` { "object": { "username": "Joe Do", "firstname": "Joe", "lastname": "Do", "_meta": { "guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "expiry": -1, "entity": 1, "removed": 0, "created": 1706396400, "updated": 1706482800, "ums": 1706482800417, "cms": 1706396400362, "update_hash": "8fc1d0b6a4e37225b1904c7de0f5a8c6431bb27e" } }, "revisions": [ { "key": "username", "value": "Joe Do", "previous_value": "Joe" }, { "key": "firstname", "value": "Joe", "previous_value": null }, { "key": "lastname", "value": "Do", "previous_value": null } ], "fields_modified": [ "username", "firstname", "lastname" ] } ``` ## Document Fast Updated Flow It's worth noting that when updates occur at high speed, the triggered events may be consolidated. Below is an example of such a combined event. If a key has been modified multiple times, it will appear multiple times in the `revisions` array. ``` { "username": "Joe", "firstname": "Joe", "_meta": { "guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "expiry": -1, "entity": 1, "removed": 0, "created": 1706396400, "updated": 1706482800, "ums": 1706482800417, "cms": 1706396400362, "update_hash": "8fc1d0b6a4e37225b1904c7de0f5a8c6431bb27e" } } ``` ``` { "username": "Joe", "lastname": "Do", "_meta": { "guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "expiry": -1, "entity": 1, "removed": 0, "created": 1706396400, "updated": 1706482800, "ums": 1706482800419, "cms": 1706396400362, "update_hash": "b7440ac2915de6f038c1a55eb9376d024f8ce1a3" } } ``` ``` { "object": { "username": "Joe", "firstname": "Joe", "lastname": "Do", "_meta": { "guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "expiry": -1, "entity": 1, "removed": 0, "created": 1706396400, "updated": 1706482800, "ums": 1706482800419, "cms": 1706396400362, "update_hash": "b7440ac2915de6f038c1a55eb9376d024f8ce1a3" } }, "revisions": [ { "key": "firstname", "value": "Joe", "previous_value": null }, { "key": "lastname", "value": "Do", "previous_value": null } ], "fields_modified": [ "firstname", "lastname" ] } ``` ## Frequently asked **Which storage events does RUAL support?** Four: on_created when a new document is created, on_updated when an existing document is updated, on_saved for either, and on_removed when a document is marked as removed. Each fires a flow through the storage_event block. **Do RUAL storage events run in order?** Not guaranteed. Events are processed first in, first out, but multiple nodes execute them simultaneously at different speeds, so strict FIFO order cannot be guaranteed. For a hard execution order, offload the intensive work to the queue. **What data does a RUAL storage event contain?** The most recent document data, a revisions array with the key, value and previous_value of each modified field, and a fields_modified list. When updates happen at high speed, events may be consolidated into one. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Storage Examples · RUAL Documentation" description: "Worked document designs for profiles, catalogs and orders." canonical: https://docs.rual.nl/blueprints/storage-examples language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Storage Examples: Real-World Data Models Practical document models for common application scenarios: user profiles, product catalogs, orders, and activity logs: plus the query patterns and optimizations to search them at scale. The [first CRUD app tutorial](https://docs.rual.nl/getting-started/first-crud-app) showed the mechanics: mutations, composed queries, get-by-guid. This page goes into data modelling. How to structure documents for real applications, given that RUAL storage is schema-less JSON with no relationships between documents. Each example includes a sample document and the query blocks to search it. For the full storage reference. See [Storage](https://docs.rual.nl/blueprints/storage). ## User Profiles A profile is a classic read-heavy document: it is loaded on almost every request, so embed everything the profile page displays. Preferences rarely change and belong to the user, so they live inside the document as a nested object. The team the user belongs to is denormalized: `team_name` is embedded for display, `team_guid` is kept as the reference for when you need the full team document. ``` { "_meta": { "cms": 1706396400362, "created": 1706396400, "entity": 1, "expiry": -1, "guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "removed": 0, "ums": 1708393100132, "update_hash": "60e27209fa93063b5605e41605c2722ed428cae0", "updated": 1708393100 }, "display_name": "joe doe", "email": "joe@example.com", "preferences": { "language": "en", "notifications": { "email": true, "push": false }, "theme": "dark" }, "role": "editor", "team_guid": "98f7999521030bbb9c96cfaf15badf216e1adc287e2bb8b0b3414a0008fc58fa", "team_name": "content team", "user_guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc" } ``` The most common lookup is by email: for example in a login flow. Since a profile is unique per email, use [`function_search_single_result`](https://docs.rual.nl/block-types/storage/function_search_single_result) instead of a regular search: | Block | Purpose | Connects To | | --- | --- | --- | | `value_default` (email, lowercased) | The email address to look up | [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) (email field) | | [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) | Exact match on `email` | [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) | | [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) | Efficient filter context, no scoring needed | [`function_search_single_result`](https://docs.rual.nl/block-types/storage/function_search_single_result) (query pin) | Once the user is logged in, switch to [`function_get_document`](https://docs.rual.nl/block-types/storage/function_get_document) with the profile's `guid`. It is the fastest retrieval method and the easiest to cache. ## Product Catalog A product carries its variants and pricing inside the document. Variants are never displayed without their product, so splitting them into separate documents would only force extra lookups. Keep prices in cents to avoid floating-point rounding. ``` { "_meta": { "cms": 1780531200118, "created": 1780531200, "entity": 1, "expiry": -1, "guid": "c45797061f2c15a75361042244385e96419d230ab804d7fa08cd52981fd78192", "removed": 0, "ums": 1782166695204, "update_hash": "f1a9c2d47b6035e8419dc0a7532be96481cd30fa", "updated": 1782166695 }, "active": true, "brand": "rual wear", "category": "clothing/shirts", "description": "soft organic cotton tee with a regular fit", "name": "essential cotton t-shirt", "sku": "TSHIRT-BLK", "variants": [ { "color": "black", "price": 2490, "size": "s", "sku": "TSHIRT-BLK-S", "stock": 14 }, { "color": "black", "price": 2490, "size": "m", "sku": "TSHIRT-BLK-M", "stock": 32 } ] } ``` Two query patterns cover most catalog pages. Category pages filter on an exact value; the search box uses full-text search on name and description. Both combine with a filter on `active` so drafts never leak into the shop: | Pattern | Query Blocks | | --- | --- | | Category page | [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) (category = clothing/shirts) + [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) (active = true) in a [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter), sorted with [`query_sort_field`](https://docs.rual.nl/block-types/query/query_sort_field) | | Search box | [`query_bool_simple_query_string_field`](https://docs.rual.nl/block-types/query/query_bool_simple_query_string_field) (name) in [`query_bool_must`](https://docs.rual.nl/block-types/query/query_bool_must), active-filter in [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter), combined with `query_and` | | Autocomplete | [`query_bool_prefix_string_field`](https://docs.rual.nl/block-types/query/query_bool_prefix_string_field) or [`query_bool_wildcard_string_field`](https://docs.rual.nl/block-types/query/query_bool_wildcard_string_field) on `name`, with a small limit | Store `name` and `description` in lowercase and lowercase the user's input before connecting it to the query block, searches are case-sensitive. To change a variant's stock after a purchase, use [`function_update_document`](https://docs.rual.nl/block-types/storage/function_update_document_mutations) with [`mutations_increment_by_field`](https://docs.rual.nl/block-types/mutations/mutations_increment_by_field). The transaction-based update system makes concurrent decrements safe. ## Order Management An order is the strongest case for denormalization. Line items are embedded. They are created once. Never change independently, and are always shown with their order. The customer is embedded as a **snapshot**: name, email, and address at the moment of purchase. If the customer later updates their profile, the order must keep the original data for invoices and shipping labels, so do not sync these fields with a storage event. ``` { "_meta": { "cms": 1782166695403, "created": 1782166695, "entity": 1, "expiry": -1, "guid": "d56808172a3d26b86472153355496fa752ae341bc915e80b19de63a920e89203", "removed": 0, "ums": 1782166695761, "update_hash": "3d82f70b5ce1946a208bd4f6019e7c5a86bb42d1", "updated": 1782166695 }, "customer_email": "joe@example.com", "customer_guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "customer_name": "joe doe", "items": [ { "name": "essential cotton t-shirt", "price": 2490, "quantity": 2, "sku": "TSHIRT-BLK-M" } ], "order_number": "ORD-2024-0042", "status": "shipped", "status_history": [ { "at": 1706396400, "status": "placed" }, { "at": 1706396700, "status": "paid" }, { "at": 1706482800, "status": "shipped" } ], "total": 4980 } ``` Do **not** split orders and line items into two storages. Without joins, an order page would need one search for the order plus a [`function_get_documents`](https://docs.rual.nl/block-types/storage/function_get_documents) call per line item set, and every status filter would have to match across storages. Embedding keeps the whole lifecycle in one document that one query can find. Status transitions are mutations on the order document: [`mutations_set_bp_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_set_bp_field_multiple) sets the new `status`, and [`mutations_add_to_array_by_field`](https://docs.rual.nl/block-types/mutations/mutations_add_to_array_by_field) appends to `status_history`. Both go through `function_update_document`, so transitions are processed sequentially. A warehouse scan and a customer cancellation can't overwrite each other. Query open work with [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) on `status`; query a customer's orders with a term match on `customer_guid`, sorted by `_meta.created` descending. The same shape works for any status-style field: one term block per allowed value in a [`query_bool_must`](https://docs.rual.nl/block-types/query/query_bool_must) for an OR filter. ## Activity Logs Logs are append-only: documents are created. Never updated, and only ever read as filtered time ranges. Model each event as a flat document with the actor denormalized onto it: log views always show who did what, and historical logs should not change when a user renames their account. ``` { "_meta": { "created": 1706396400, "expiry": 1737759600, "guid": "b1fd2ecf516b6b225a3fb4cc729932892a45abb2a5ee5fcb40c11ef6b1b80bd6" }, "actor_email": "joe@example.com", "actor_guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "context": { "ip": "203.0.113.10", "user_agent": "mozilla/5.0" }, "event": "user.login", "severity": "info" } ``` Use `_meta.expiry` for retention instead of building a cleanup job: pass an expiry date to the `expiry` in-pin of [`function_create_document`](https://docs.rual.nl/block-types/storage/function_create_document_from_mutations): for example now plus 90 days: and storage removes the document automatically. Note that expired documents are **permanently** deleted. Time-based queries range over `_meta.created`, a Unix timestamp. Combine a range with a term filter on `event` or `actor_guid`: | Pattern | Query Blocks | | --- | --- | | Last 24 hours | [`query_bool_range_field`](https://docs.rual.nl/block-types/query/query_bool_range_field) on `_meta.created` (greater than now − 86400) in [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) | | Events of one user | [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) (actor_guid) + range on `_meta.created`, sorted descending with [`query_sort_field`](https://docs.rual.nl/block-types/query/query_sort_field) | | Errors only | [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) (severity = error) + range, combined with `query_and` | ## Search Optimization - **Store searchable values in lowercase.** Searches are case-sensitive: `Joe` will never match `joe`. Lowercase values on write, and lowercase user input before connecting it to a query block. - **Reuse cache keys.** `function_search` has a `cache key` in-pin. Identical queries with the same key. A category page, a profile lookup: are served from cache instead of hitting the index on every request. - **Paginate with limit and offset.** Connect a `number` block to the `limit` in-pin (or use `query_limit` in the query) and multiply `offset` by the page number. Always set a limit; smaller result sets are faster to return and render. - **Select only the fields you render** with [`query_source_by_field`](https://docs.rual.nl/block-types/query/query_source_by_field) for list pages that need two or three fields per document. - **Be deliberate with [`query_disabled_request_cache`](https://docs.rual.nl/block-types/query/query_disabled_request_cache).** It forces the search to wait until all pending inserts are indexed. The right tool for heavy aggregation-style queries that must see every write, but a source of noticeable delay in high-insert storages like activity logs. For counters and statistics, prefer maintaining a counter field with [`mutations_increment_by_field`](https://docs.rual.nl/block-types/mutations/mutations_increment_by_field) or aggregating periodically into Redis; see [Common Pitfalls](https://docs.rual.nl/blueprints/common-pitfalls). #### Denormalize vs Reference by GUID | Denormalize (embed) when… | Reference by guid when… | | --- | --- | | You display the data in every list or detail view (`customer_name` on an order). | You only need the full document on a detail view (`team_guid` on a profile). | | The data is a historical snapshot that must not change (order customer, log actor). | The related data changes often and staleness is not acceptable. | | You filter or sort on the value in search queries. | The embedded payload would be large or deeply nested. | Even when you embed. Always keep the related `guid` on the document. It is your way back to the source with [`function_get_document`](https://docs.rual.nl/block-types/storage/function_get_document), and the key a [storage event](https://docs.rual.nl/blueprints/storage-events) uses to find and re-sync embedded copies when syncing *is* what you want. - [Storage](https://docs.rual.nl/blueprints/storage): The full storage reference: meta data, query types, lifecycle, expiry, and caching. - [Storage Events](https://docs.rual.nl/blueprints/storage-events): Keep denormalized copies in sync when a source document changes. - [First CRUD App](https://docs.rual.nl/getting-started/first-crud-app): New to storage? Build the task manager tutorial first, then return here. ## Frequently asked **Should I embed or reference related data in RUAL storage?** Denormalize when you display the data in every list or detail view, when it is a historical snapshot that must not change, or when you filter and sort on it in queries. Reference by guid when you only need the full document on a detail view, when the related data changes often, or when the embedded payload would be large. Even when you embed, always keep the related guid on the document. **How do I auto-delete old documents in RUAL storage?** Pass an expiry date to the expiry in-pin of function_create_document, for example now plus 90 days, and storage removes the document automatically, no cleanup job needed. Note that expired documents are permanently deleted. **How do I query RUAL documents created in the last 24 hours?** Use query_bool_range_field on _meta.created, a Unix timestamp, with a range greater than now minus 86400 inside a query_bool_filter, and combine it with term filters on fields like event or actor_guid through query_and. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Protected Storages · RUAL Documentation" description: "Keep a storage usable by a flow and by your users while its contents stay out of the studio." canonical: https://docs.rual.nl/blueprints/protected-storages language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Protected Storages Some data has to be usable by a flow without being readable by the person building that flow. A protected storage keeps working for your users and stops handing its contents to the studio. Building a flow and reading the data that flow touches used to be the same permission. To check your work you press simulate, and simulating shows you what came back: that is the whole point of it. For a `tasks` storage that is exactly right. For `invoices`, `salaries` or a case file it is not, because the developer wiring the reminder email has no business paging through every invoice in the company to do it. Protecting a storage separates those two things. The flow keeps working, end to end, for the people it was built for. The contents stop arriving on the developer's screen. ## What Protection Actually Does When a protected storage is read by a developer, the documents still come back, but only as outlines: the identity and the timestamps that make a flow work, and a marker saying the content was withheld. Nothing else. The marker matters more than it sounds: without it, a withheld document and a document that does not exist look identical, and a flow cannot tell "the customer has no invoices" from "you are not allowed to see them". Totals and groupings return nothing at all. There is no point withholding the amount on every invoice and then answering "what is the sum of the amounts", or hiding the customer name and answering "count these by customer". Anything that summarises a protected storage comes back empty for the same reader the outlines are for. ## What still works Selecting the storage on a block, wiring it, writing to it, reading its field definitions, and running the whole flow. Every document the storage holds is still delivered in full to the user whose flow it is. Protection changes what the builder sees, not what the application does. ## What is withheld Document contents in a studio run, the studio console output of any run that touched the storage, and every total, average or grouping over it. You get the shape of the answer and never the values inside it. ## Who Sees the Real Documents The rule is about who is at the screen, not which blocks are on the canvas. A run started from the studio is a developer looking at output; a run started by a user is an application doing its job. | Who is running the flow | What they get back | | --- | --- | | A developer pressing simulate | Outlines | | A developer simulating *as* another user | Outlines. Simulating as somebody else does not move the screen. | | A developer starting a production run from the studio | Outlines. Same developer, same screen, same answer. | | A user calling your endpoint or opening your page | The real documents | | A scheduled run, a storage event or a queued job | The real documents | | A cluster admin | The real documents | | Anyone holding the protected read scope | The real documents | The last two rows are the deliberate exemptions. A cluster admin already administers the cluster the data sits on, so withholding it from them would be theatre rather than protection. The scope exists so that one person can be given the exemption for an afternoon of debugging without being made an administrator of everything else. ## The Protection Follows the Data, Not the Screen A mask that only covers the obvious read is worth very little, because the developer it excludes is by definition someone who can write blocks. Four things close the routes around it. | The way around it | What happens instead | | --- | --- | | Read the invoice, then log it to the console | A run that touched a protected storage has its console output withheld for the rest of that run. The flow still runs and still logs; the studio just does not print it. | | Read the change history instead of the document | Revision history inherits the protection of the storage it is the history of. A protected storage has a protected history, which is the only sensible reading: a revision holds the values before and after, so an unprotected history is the document twice over. | | Push the documents through the queue and read them out of the job | The mark travels with the job. The flow that read the invoice and the worker that processes it are two separate runs, so the tag is carried across the boundary and re-applied to whatever waits on the result. | | Refer to the storage by a different spelling of its name | Nothing changes. The flag is attached to the data, not to one way of writing its name, so capitalisation and punctuation in the name make no difference to what is protected. | One more rule sits underneath all of these: when the platform cannot determine whether a storage is protected, it treats it as protected. A lookup that fails withholds the data. The cost of being wrong in that direction is a developer waiting a few seconds; the cost of being wrong in the other direction is the invoices. > **The queue result is deliberately readable** A flow waiting on a queued job can read what that job returned, otherwise the job would be pointless. What closes is the developer's window onto the run, which is the same rule the rest of the feature follows. ## Turning It On Only a cluster admin can protect a storage or unprotect it, and that is a deliberate narrowing rather than an oversight. The permission that governs the other storage settings is exactly the permission the excluded developers already hold, so it could not be the permission that guards this one: anyone able to turn it on would be able to turn it off again the moment it got in their way. The switch lives on the storage itself, next to its other options in RUAL Studio, and takes effect within seconds. There is no migration, no reindex and no downtime: the documents do not move, only what is handed back changes. Turning it off again is just as immediate, which makes it safe to try on a copy first. ## Building Against a Protected Storage You are building blind on values, not on structure, so the working method shifts a little. | Instead of | Do this | | --- | --- | | Reading real rows to learn the document shape | Read the field definitions. They are the schema and they are not the data, so they stay fully visible. | | Simulating against production data | Keep an unprotected storage of invented rows with the same fields, and point the storage block at it while you build. Switch it over when the flow is right. | | Checking a value to see whether the read worked | Check the flow. A read that found a document continues down the found path and carries the withheld marker; a read that found nothing does not. | | Adding a total to the console to sanity check an aggregation | Build the aggregation against the test storage. Over a protected storage it is empty by design, and that is not a bug you can debug your way out of. | | Asking for the protection to be lifted permanently | Ask a cluster admin to run it once and tell you what came back, or to hand you the exemption scope for the session. Both leave a trail; a permanently unprotected storage does not. | ## What It Is Not Protection is one specific thing, and reading it as more than that is how people end up surprised. - **It is not a permission system for your own application.** If you write an endpoint that returns every invoice to anyone who calls it, protection will not stop it: your user's run is exactly the run that gets the real documents. Endpoint access is still yours to control with [scopes and rate limits](https://docs.rual.nl/blueprints/remote-access-control). - **It is not encryption.** It governs what the platform hands to a reader. It is a decision about who receives data, not a transformation of the data at rest. - **It is not a hidden storage.** Developers still see the storage exists, what fields it has and how many documents it holds. That is what lets them build against it at all. - **It does not stop writes.** A flow can still create and update documents in a protected storage. Withholding what an update returns would break the update itself, so what comes back afterwards is masked, not the write. ## Which Storages Deserve It The useful test: would you export this storage to a spreadsheet and email it to the person building the flow? If the answer is no, protect it. | Storage | Protect | Why | | --- | --- | --- | | Invoices, orders, payments | Yes | Commercially sensitive in aggregate even when no single row is dramatic. The totals are usually the part you least want browsed. | | Salaries, contracts, HR records | Yes | The classic case where a colleague building the flow is specifically the person who should not read it. | | Medical, case or client notes | Yes | Usually covered by an agreement that names who may see the contents. A build permission is not that name. | | Identity documents, bank details | Yes | Nothing about developing a flow requires seeing them once. | | Products, pages, settings, tasks | No | Reading them is how you build against them, and there is nothing to withhold. | | Anything you are still designing | Not yet | Protect it once the shape is settled. Designing a document you cannot see is unnecessarily hard. | > **Protecting the credentials, not just the rows** A storage holds your data; an integration holds someone's key. The same idea applies there, and it is covered on [Service Providers](https://docs.rual.nl/blueprints/service-providers): a blueprint names the provider and never carries the secret. ## Next Steps ## Frequently asked **How do I stop developers reading a storage in RUAL?** Mark the storage as protected against developer read. A cluster admin sets it on the storage itself. After that a run started from the studio gets each document as an outline: its identity, its timestamps and a marker saying the content was withheld. Runs started by your users, by a schedule, by a storage event or by the queue keep receiving the real documents. **Does a protected RUAL storage still work for end users?** Yes. Protection changes what the person building the flow sees, not what the application does. A user opening a page or calling an endpoint receives the full documents, and flows can still create and update documents in a protected storage. **Why does an aggregation over a protected RUAL storage return nothing?** By design. Withholding the amount on every invoice and then answering a question about the sum of the amounts would hand over exactly what the outline withholds, so totals, averages and groupings come back empty for the same reader. Build aggregations against an unprotected test storage. **Who can read a protected storage in RUAL?** The user whose own run it is, any scheduled or queued run, a cluster admin, and anyone holding the protected read scope. That scope exists so one person can be given the exemption for a debugging session without being made an administrator of everything else. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Repeating Events · RUAL Documentation" description: "Schedule work on an interval and keep repeated runs from overlapping." canonical: https://docs.rual.nl/blueprints/repeating-events language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Repeating Events Within blueprints, it is possible to schedule events to run at specified time intervals. The minimum frequency of these events may vary depending on the license, with the potential to run as frequently as every second. ### The repeating event block A repeating event in RUAL functions similarly to the [setInterval()](https://developer.mozilla.org/en-US/docs/Web/API/setInterval) function in JavaScript or [crontab](https://man7.org/linux/man-pages/man5/crontab.5.html) in Linux. However, in this case, you specify a starting date. When the block is executed, it evaluates the input pin `Next Run` and utilizes its output to create a new event in the [queue](https://docs.rual.nl/blueprints/queue). You can monitor the currently scheduled repeating events in the queue overview within the cluster. The system checks whether the repeating event is still active when the associated blueprint is activated or deactivated. The execution of the repeating event's flow occurs at the specified time, taking into account the maximum concurrent queue limit imposed by your license. If you have more queue items running simultaneously, they will automatically be postponed. Additional information about [auto-postpone](https://docs.rual.nl/blueprints/queue#auto-postpone) can be found for reference. ## Event not executing There may be instances where it appears that the repeating event is not being executed. This could be due to its rapid execution, causing it to seem as if it has already occurred. Additionally, forgetting to activate the blueprint after creating the event can lead to this issue. It is also crucial to ensure that the `Next Run` input pin is properly filled, as failure to meet the specified date and time requirements could result in the event being canceled by the system. ## Requirements For a repeating event to function as expected, the following requirements must be met. Additionally, it's important to note that a queue item will cease execution once it reaches the maximum `Execution Time` specified within the license. - The `Start At` pin must be populated and set to a future date and time. - The `Next Run` pin must be populated and set to a future date and time. - The value of the `Next Run` pin must be equal to or greater than the `Lowest Queue` time specified in the license. ## How to Create a Repeating Event To create a repeating event, either open a new Blueprint or access an existing one, then `right-click` on the canvas. Select the `Add New Block` option, which will display a flow dialog. In this dialog, search for `repeating event` and select the blue block labeled with the main group `events`. Once the [`repeating event`](https://docs.rual.nl/block-types/events/schedule_repeating_event) block is created, it presents two in-pins: `Start At` for the initial date and `Next Run` for subsequent events. `Right-click` and drag from the `Start At` pin to add the [`current date now`](https://docs.rual.nl/block-types/date%2Ctimezone/date_currentdate) block, which is commonly used as the starting date, though you can specify a future date if needed. For the `Next Run` pin, start by right-click dragging from the date pin and dropping it elsewhere on the canvas. Then, disconnect and left-click drag from the `date now` block to open a dialog where you can search for operations like `plus seconds` within the [date group](https://docs.rual.nl/block-types/date?group=date). Connect the `Seconds` in-pin to the [`custom number`](https://docs.rual.nl/block-types/number/number_default) block and set it to `60`. Upon deploying and activating the blueprint, a queue item for this repeating event is created, and it will execute every `60` seconds. ## Frequently asked **How do I run a RUAL flow on a schedule?** Use the schedule_repeating_event block, which works like setInterval or crontab. Its Start At pin sets the initial run and its Next Run pin schedules the following one: when the block executes, it creates a new event in the queue for the next run. **Why is my RUAL repeating event not executing?** The usual causes are that the blueprint was never activated after the event was created, or that the Next Run pin is not filled with a future date and time, which lets the system cancel the event. Both Start At and Next Run must hold future dates, and Next Run must respect the lowest queue time of your license. **How often can a RUAL repeating event run?** The minimum frequency depends on your license, with the potential to run as often as every second. Execution is subject to the license's maximum concurrent queue limit, excess items are automatically postponed, and a queue item stops once it reaches the maximum execution time. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Queue · RUAL Documentation" description: "Hand slow or bursty work to the queue and track what each task did." canonical: https://docs.rual.nl/blueprints/queue language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Queue / Task manager We offer a queue feature in our system, and it's important to understand when and how to effectively utilize it. ### Understanding Queues For large-scale system development, offloading work to other nodes is essential. Manually managing this process can be cumbersome. To encourage efficient offloading, we've streamlined the use of our queue system, making it as user-friendly as possible. ## Appropriate use cases One might wonder when and why to utilize a queue. Well, certain tasks can simply be deferred for later execution. An obvious and frequently encountered example is the generation of PDF documents, which is a computationally intensive task that doesn't necessarily need to be performed immediately. The timeliness of having the PDF ready is not critical enough to keep the user waiting. Instead, we create a custom function responsible for generating the PDF and notifying the user via email. If we were to request the PDF generation within the same workflow, it would cause the user to experience delays. Therefore, we aim to decouple the PDF generation from the main workflow, and this is precisely where a queue item becomes essential. ## How to utilize The queue block is designed to execute as efficiently as possible. You provide it with the function you wish to execute, and it will automatically run that function in the background. It may run on another node or thread, as the queue's purpose is to abstract away the specific execution location, aiming to execute as closely to the specified time as feasible. You have various options at your disposal to customize the behavior of the queue block. Detailed descriptions of each input pin can be found in the [block documentation](https://docs.rual.nl/block-types/function%20execution/function_custom_execute_from_queue). ## Auto Postpone execution Each license comes with a predefined limit for the maximum number of concurrent queue items that can be executed simultaneously. This limit serves a dual purpose: it allows you to build high-end systems with fast-running, high-performance functions, while also preventing the system from becoming overly congested with running queue items. When you create multiple queue items simultaneously, the system automatically adds a few seconds to the execution time of each item to facilitate a more even distribution of workload. ## Monitoring and Analytics When managing queue items, it's important to have insight into their execution status and timing. You may wish to ascertain whether they have executed and when these executions occurred. Within the [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio) interface, you can access the `Queue` option in the left-side menu. Navigating to this page will provide you with a comprehensive list of currently executing, scheduled, and completed queue items. The data displayed on this page is quasi-real-time. As queue items progress through their lifecycle, you will observe them transitioning between tables, though there may be a slight delay of a few seconds. The data is retrieved directly from our backend, but it is not updated in real-time to ensure optimal system performance. ## Frequently asked **When should I use the queue in RUAL?** Use the queue for work that is slow or bursty, so the calling flow does not wait for it. Tasks are handed to the queue and executed separately, and you can monitor what each task did. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Assets · RUAL Documentation" description: "Upload, manage and serve files, and reference them from a blueprint." canonical: https://docs.rual.nl/blueprints/assets language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Assets: Files in Your Cluster Store files on your cluster and serve them anywhere: public assets on the `/static/` path, private assets behind scopes: uploaded in RUAL Studio or managed through the asset APIs. ## What Are Assets? Assets are files stored centrally on your cluster. Because blueprints are distributed across multiple nodes hosted in various regions, keeping files next to a single node does not work: assets solve this by making every uploaded file available to all your blueprints and all your connected domains, with automatic storage and robust security measures. There are two kinds of assets: - **Public assets** live in the `public` folder. They are served over HTTP from every connected domain under the `/static/` path: no login or token required. - **Private assets** live anywhere **except** the `public` folder. They are never served directly over HTTP; only your blueprints can read them, so you decide exactly who gets access. Typical uses are images and fonts for your user interfaces, downloadable files such as exports and invoices, and generated reports you want to keep behind a login. All public files, regardless of their format, are accessible from all your connected domains using the `/static/` path, which maps to the root of the `public` folder. Referring to the file tree below, the stylesheet is served at `/static/styles/mobile.css`, while the report in `system_assets` has no public URL at all. ## Uploading Assets in RUAL Studio The **Manage Assets** section in [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio) is where you organize folders, upload files, and inspect what is stored on the cluster. - Open RUAL Studio and go to **Manage Assets**. You see the asset folder tree, including the `public` folder, and the files inside the selected folder. - Select the folder you want to upload to: or create a new subfolder first. Remember: anything placed inside `public` becomes publicly available. - Click **Upload** and pick one or more files from your device. - Confirm the upload. The files appear in the file list and are stored on the cluster immediately. - Select an uploaded file to open its detail view, where you can review the file metadata and, for public assets, copy the `/static/` URL. > **Public Means Public** Everything inside the `public` folder is world-readable on every connected domain. Never store files containing personal data, credentials, or other sensitive content there. Keep those in a private folder and serve them through a scoped blueprint, as described in [Serving Protected Files](#serving-protected-files). ## Managing Assets with the API For scripted uploads, integrations, and housekeeping, the cluster ships built-in asset APIs under the **System Assets** category. All of them require an `access_token`. See [Getting Access](https://docs.rual.nl/cluster/getting-access): and the token's user needs the matching scope: `assets_create` to upload and search, `assets_remove` to delete. | Endpoint | Purpose | Required scope | | --- | --- | --- | | `POST /api/v1/_system/assets/stream` | Upload a file with streaming support (max 500MB) | `assets_create` | | `POST /api/v1/_system/assets/search` | Search and list uploaded assets | `assets_create` | | `DELETE /api/v1/_system/assets/{guid}` | Delete an asset from filesystem and database | `assets_remove` | #### Uploading a File The upload endpoint takes the file as `multipart/form-data` and the metadata as query parameters: | Parameter | Required | Description | | --- | --- | --- | | `filename` | yes | Original filename, URL encoded. | | `filesize` | yes | File size in bytes; the maximum is 500MB. Larger uploads are rejected with `FILE_TOO_BIG`. | | `directory` | yes | Target directory: `public`, `system_assets`, or `temp`. | | `filetype` | no | MIME type. When omitted, it is detected from the file extension. | | `path` | no | Custom subdirectory below the target directory, URL encoded. | | `action` | no | Blueprint action GUID; creates a blueprints subfolder for the upload. | ``` curl -X POST "https:///api/v1/_system/assets/stream?filename=mobile.css&filesize=4821&directory=public&path=styles" \ -H "Authorization: Bearer " \ -F "file=@mobile.css" ``` ``` { "assets_guid": "9f8e7d6c5b4a3210", "name": "mobile.css", "type": "text/css" } ``` The returned `assets_guid` identifies the stored asset. Keep it when you want to delete the file later. Uploading to `directory=public` makes the file available at `/static/ / ` right away; `system_assets` keeps it private. #### Searching Assets The search endpoint accepts a query string with paging, so you can list folders or find a specific file: ``` { "limit": 25, "offset": 0, "query": "logo" } ``` Set `"count": true` to get the number of matching assets instead of the results. `limit` defaults to 50 and accepts up to 1000. #### Deleting an Asset ``` curl -X DELETE "https:///api/v1/_system/assets/9f8e7d6c5b4a3210" \ -H "Authorization: Bearer " ``` A successful delete returns the removed `guid`; an unknown guid returns `404`. The full request and response schemas live under [System Assets APIs](https://docs.rual.nl/cluster/api/system-assets). ## Public vs Private Assets | | Public | Private | | --- | --- | --- | | **Location** | The `public` folder (API: `directory=public`) | Any folder outside `public` (API default: `system_assets`) | | **URL pattern** | `https:// /static/ ` on every connected domain | No direct URL: readable only by blueprints | | **Who can access** | Everyone, without login or token | Only the flows you build; gate them with `*loggedin` or custom scopes | | **Typical use** | Images, stylesheets, fonts, public downloads | Internal documents, user-specific exports, reports | The rule of thumb is simple: the `public` folder decides everything. A file inside it is public no matter what; a file outside it can never be reached by URL guessing, because the `/static/` path only maps into `public`. ## Serving Protected Files To give selected users access to a private asset, serve it through a blueprint and put access control on that blueprint: - Create an API or page that loads the file with the [get file from assets](#assets-in-blueprints) block and returns it to the caller. - Click the lock icon on the block and assign `*loggedin`, any logged-in user: or a custom scope such as `reports`, so only users carrying that scope can call it. See [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control) for the full scoping tutorial. - Users without the scope are redirected to the forbidden access page (`/403`). For downloads straight from the browser. An ` ` tag cannot set an `Authorization` header. Pass the token in the URL instead: `?access_token= `. Tokens are valid for 14 days and extend automatically while actively used; all five ways to provide a token are documented in [Adding tokens to your requests](https://docs.rual.nl/cluster/api-guide#token-providing). ## Using Assets in Blueprints The [`get file from assets`](https://docs.rual.nl/block-types/files/file_from_assets) block (type [`file_from_assets`](https://docs.rual.nl/block-types/files/file_from_assets), searchable by its alias `assets`, introduced in v10.4.28) makes files from assets accessible in blueprints. Give it the local path and it hands you a `file` you can show in the UI or return from an API: - **In-pin: asset path** (`value`, required): the local path to the asset, e.g. `/public/images/image.png` or a private path outside the public folder. - **Out-pin: success** (`condition`): the asset exists. - **Out-pin: file** (`file`): the file, ready to be used by UI blocks or API responses. - **Out-pins. Name, type, size** (`value`, `value`, `number`): the filename, the asset type, and the size in bytes. Public assets can be referenced directly in the UI through their `/static/` URL without any block. The block earns its place with private files: it is the only way to pull them into a flow, which is exactly what keeps them protected. ## Organizing Folders A consistent folder layout pays off once assets pile up. Recommended conventions: - **Group by purpose**, not by date or uploader: `public/styles`, `public/images`, `public/downloads` for public files, and folders like `reports` or `exports` next to. Never inside: `public` for private ones. - **Name for URLs**: use lowercase names with hyphens instead of spaces. Folder and file names below `public` become part of the `/static/` URL verbatim. - **Remember the prefix mapping**: `/static/` points at the root of `public`, so `public/styles/mobile.css` is served as `/static/styles/mobile.css`. The `public` segment itself never appears in the URL. - **Keep user-generated content out of public** unless it is genuinely meant for everyone; move it through a scoped blueprint instead. ## File Size, Formats & Performance - **Size limit**: the upload API accepts files up to **500MB** per file and rejects larger ones with `FILE_TOO_BIG`. For anything near that limit, prefer the streaming endpoint over base64-style uploads. - **Image formats**: serve images as `webp`: dramatically smaller than PNG or JPEG at the same quality. The [`function_image_to_webp`](https://docs.rual.nl/block-types/files/function_image_to_webp) and [`function_resize_image`](https://docs.rual.nl/block-types/files/function_resize_image) blocks convert and resize images in a flow, so you can normalize user uploads before storing them. - **Documents**: distribute documents as PDF; it renders consistently on every device and browser. - **Immutable URLs**: public assets are ideal CDN and browser-cache candidates because every connected domain serves the same bytes. Give files that change over time a version in the filename: `app.v3.css` instead of `app.css`: so clients can cache each version aggressively and you never wait for stale caches to expire. Upload a new name, deploy, and the new URL takes effect immediately. ## Working With Video Video is the one upload type where storing what arrived is usually the wrong move: a phone clip is often twenty times larger than it needs to be, and serving it as uploaded costs bandwidth on every view. Three blocks cover the normal pipeline, and the split between them matters because two are cheap and one is not. | Block | What it does | Where it runs | | --- | --- | --- | | [`video_probe`](https://docs.rual.nl/block-types/video/video_probe) | Reads width, height, duration, size and whether the file has an audio track. It reads the file's own header rather than its contents, so nothing is decoded. | Inline, in milliseconds | | [`video_thumbnail`](https://docs.rual.nl/block-types/video/video_thumbnail) | Writes one frame out as a JPEG poster image. Pick the timestamp; a timestamp past the end of the clip falls back to the midpoint rather than failing. | Inline: it decodes a single frame, not the file | | [`video_compress`](https://docs.rual.nl/block-types/video/video_compress) | Re-encodes the video, optionally scaling, capping the bitrate and trimming to a range. Presets cover the normal message-media sizes; every setting can be overridden individually. | The [queue](https://docs.rual.nl/blueprints/queue), as a background job | Probe first and branch on the answer. A clip already inside your limits does not need re-encoding, and skipping the compression job is always faster than running a good one. - **Compression is a queue job, not a step.** The block hands you the job id and the path the output will appear at, and the flow continues. Wire the `on_complete_function` and `on_error_function` out-pins to react to the outcome; both fire whether or not the flow waited. - **Waiting in the flow is available and rarely what you want.** Setting `wait_in_flow` blocks until the encode finishes and fills in the output dimensions, duration and size. It is convenient in a scheduled job and a poor idea in a request: a user is waiting on a page while a video transcodes. Give it a timeout when you do use it. - **The result out-pins are empty unless you waited.** Width, height, duration, size and success are populated by the job, so a flow that did not wait reads them from the callback instead. - **Both file pins take either shape.** A file object from another file block, or a path relative to the entity root. Thumbnails written below `public` are servable straight from the `/static/` URL. - **Trim to enforce a maximum length, do not check afterwards.** The start and duration pins cut the clip as part of the encode. Without them a twenty-minute upload is transcoded in full and can only be rejected once the work is already done, which is the expensive order to discover it in. - **Pass context through rather than storing it.** The `passthrough` in-pin is handed to the callback verbatim, which is how the completion function knows which document the clip belonged to. ## Next Steps - [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control): Gate pages and APIs behind *loggedin or custom scopes. - [System Assets APIs](https://docs.rual.nl/cluster/api/system-assets): Full request and response schemas for the asset upload, search, and delete endpoints. - [get file from assets](https://docs.rual.nl/block-types/files/file_from_assets): Pin reference for the block that loads cluster assets into a blueprint. ## Frequently asked **How do I serve a file from RUAL?** Upload it as an asset in RUAL Studio or through the asset APIs. Public assets are served from the /static/ path, and private assets stay behind scopes so only authorised callers can fetch them. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "System Settings · RUAL Documentation" description: "Read and write cluster-wide settings from a blueprint." canonical: https://docs.rual.nl/blueprints/system-settings language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # System Settings Utilizing System Settings for global management of configuration parameters. ### Managing Central Configuration Parameters System settings serve as a centralized method for globally managing cluster configuration parameters throughout the entire cluster. We have already established certain configuration parameters and provided documentation regarding their purpose and significance. You can incorporate system settings within blueprints, either through static selections or dynamic values. Typical use cases for system settings include managing email tokens, establishing global reply-to addresses, or configuring global parameters for various system components. ## Securing Values We also offer the option to secure values. When you create a secure value, it remains concealed and is not directly visible when accessing system keys or reviewing them in overviews or blueprints. Any request to view a secured system setting is logged for audit trail purposes. This way, you can track who accessed which tokens, providing you with the ability to manage token access and revocation effectively. ## Default Configuration Keys By default, the following keys are predefined and can be modified to alter the behavior of the cluster. Each key's purpose, example value, and when to change it is covered in detail in the [System Settings Reference](https://docs.rual.nl/blueprints/system-settings-reference), including secure token handling and a complete email configuration example. | Key | Description | | --- | --- | | `timezone` | Specifies the default timezone for the execution of `date` type blocks. | | `thousand_separator` | Determines the default separator used for formatting thousands in numbers. | | `decimal_separator` | Specifies the default separator used for formatting decimals in numbers. | | `language_user_enabled` | If set to `true`, the user's data will be used to determine their language preference. | | `language_query_enabled` | If set to `true`, the data from the `query` will be used to determine the user's language. | | `language_browser_enabled` | If set to `true`, the data from the `supported language` in the browser will be used to determine the user's language. | | `language_default` | Specifies the default language to use when the user's language cannot be determined or does not match a supported language. | | `input_datetime_format` | Defines the date format to be used in datetime inputs. | | `input_date_format` | Specifies the date format to be used in date inputs. | | `date_default_format` | Specifies the default date format used when rendering dates in the user interface. | | `default_divide_by` | Specifies the default number by which to divide `pricing` inputs, for example, `100` to store pricing fields in cents. | | `email_default_from` | Determines the default value to be used as the `from` address when sending emails. | | `messagebird_originator` | Specifies the default originator name used in [MessageBird](https://messagebird.com/) SMS for two-factor authentication (2FA). | | `postmark_token` | Determines the default [server token](https://postmarkapp.com/) to be used when sending password reset emails. | ## Frequently asked **What are RUAL system settings used for?** They are the centralized method for managing cluster-wide configuration parameters, from timezone and date formats to email tokens and global reply-to addresses. Blueprints read them through the get system setting block, either as static selections or as dynamic values. **How do I store a secret like an API token in RUAL?** Create the system setting as a secure value. It stays concealed when accessing system keys, in overviews and in blueprints, and any request to view a secured setting is logged for audit purposes, so you can track who accessed which token. **Which system settings does RUAL predefine?** Keys for timezone, thousand and decimal separators, the four language settings, date and datetime input formats, the default date rendering format, default_divide_by for pricing inputs, email_default_from, messagebird_originator and postmark_token. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "System Settings Reference · RUAL Documentation" description: "Every system setting, its type and what changing it affects." canonical: https://docs.rual.nl/blueprints/system-settings-reference language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # System Settings Reference Every predefined system setting explained: purpose, example value, when to change it, and how to handle secure tokens. System settings are the cluster's central configuration store: predefined keys that tune cluster behavior, plus your own custom keys for integration tokens and environment parameters. This reference covers every predefined key, when a change is justified, and how to work with secure values. For the conceptual overview. See [System Settings](https://docs.rual.nl/blueprints/system-settings). ## Predefined Keys These keys exist on every cluster. Changing one alters cluster behavior immediately, no deploy needed. | Key | Purpose | Example | When to change it | | --- | --- | --- | --- | | `timezone` | Default timezone for executing `date` type blocks. | `Europe/Amsterdam` | When your users and data live in a different timezone than the cluster default. | | `thousand_separator` | Separator for formatting thousands in numbers. | `.` (10.000) or `,` (10,000) | To match your locale's number display. | | `decimal_separator` | Separator for formatting decimals in numbers. | `,` (9,99) or `.` (9.99) | To match your locale's number display: pair it with `thousand_separator`. | | `language_user_enabled` | Uses the user's profile to determine their language. | `true` | When users have mixed languages and you store a preference per user. | | `language_query_enabled` | Uses the query string to determine the user's language. | `true` | When you pass `?lang=` (or similar) in shared links. | | `language_browser_enabled` | Uses the browser's supported language to determine the user's language. | `true` | Usually left on; turn off to force `language_default` for everyone. | | `language_default` | Fallback language when no preference can be determined. | `nl` or `en` | Set once, at cluster setup, to your primary audience's language. | | `input_datetime_format` | Date format used in datetime inputs. | `YYYY-MM-DD HH:mm:ss` | When users expect a different entry format (e.g. US month-first). | | `input_date_format` | Date format used in date inputs. | `YYYY-MM-DD` | When users expect a different entry format. | | `date_default_format` | Default format for rendering dates in the UI. | `YYYY-MM-DD HH:mm:ss` | To change how dates appear throughout your pages. | | `default_divide_by` | Default divisor for `pricing` inputs. | `100` (store cents) | Set to `100` when prices are entered as whole euros but stored as cents, prevents rounding errors. | | `email_default_from` | Default `from` address for outgoing emails. | `noreply@yourdomain.com` | Required before any mail leaves the cluster. See the email setup below. | | `messagebird_originator` | Default originator name for [MessageBird](https://messagebird.com/) SMS used in two-factor authentication. | `YourApp` | Only when you use SMS-based 2FA via MessageBird. | | `postmark_token` | Default [Postmark server token](https://postmarkapp.com/) for sending emails (password resets, notifications). | `xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx` | Once per cluster, before sending email. **Store it as a secure value**. It is a secret. | ## When to Modify Settings Change a predefined key only when the default actively hurts. Legitimate reasons: - **Localization**: timezone, separators, languages, and date formats should match your audience (once, at setup). - **Integrations**: tokens and originators for email/SMS must exist before the features that use them work. - **Environment differences**. Use custom keys for values that differ between development and production (API hosts, webhook URLs), so the same blueprint works on both by reading the key. Avoid using settings as a general data store: documents belong in a [storage](https://docs.rual.nl/blueprints/storage), configuration of behavior belongs in settings. ## Secure vs Non-Secure Values A value marked **secure** stays concealed: it is not shown in overviews, in the Studio, or in blueprint data. Every read of a secured key is written to the audit log, so you can see exactly who accessed which token and revoke accordingly. | | Non-secure | Secure | | --- | --- | --- | | Visible in Studio / overviews | Yes | No, concealed everywhere | | Readable by blueprints | Yes | Yes. The value is injected at execution time | | Reads logged in audit log | No | Yes | | Use for | Formats, locales, feature flags, public hosts | API tokens, passwords, signing secrets (e.g. `postmark_token`) | Control who can manage settings at all with the `setting_manage_users`-family permissions. See [User Access Management](https://docs.rual.nl/cluster/user-access-management). ## Email Configuration Example A complete working email setup needs two keys: - `postmark_token`. Create a server token in Postmark, add it as a **secure** system setting. - `email_default_from`. Set it to an address on your verified Postmark sender domain, e.g. `noreply@yourdomain.com`. From that point, password resets and any email-sending blocks work without further configuration. For SMS-based two-factor authentication, also set `messagebird_originator` to your sender name. ## Custom Keys & Token Management You can add your own keys. Conventions that keep them manageable: - **Naming**: prefix by integration: `stripe_secret_key`, `slack_webhook_url`. Keep lowercase with underscores. - **Secrets**. Always mark tokens secure at creation time; rotating means editing the value in one place, and every blueprint picks it up on the next run. - **Custom vs predefined**: predefined keys tune the cluster itself; custom keys hold your integration parameters. Do not overload a predefined key with an unrelated value. ## Reading Settings in Blueprints Use the `get system setting` block to read a key into a flow. Pick a predefined key statically or provide the key name dynamically as a value. The block returns the current value at execution time, including decrypted secure values (never logged in plain text by the platform). Typical pattern: read `stripe_secret_key` at the start of a payment flow, then pass it into the request blocks. One setting to rotate, zero blueprint edits. ## Next Steps - [System Settings overview](https://docs.rual.nl/blueprints/system-settings): The conceptual introduction to system settings and securing values. - [User Access Management](https://docs.rual.nl/cluster/user-access-management): Control who may manage settings and read secure values. - [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control): Scopes and access rules for APIs built with tokens from settings. ## Frequently asked **Do RUAL system setting changes need a deploy?** No. Changing a predefined key alters cluster behavior immediately. Blueprints read a key with the get system setting block, which returns the current value at execution time, including decrypted secure values. **What is the difference between a secure and a non-secure system setting in RUAL?** A secure value is concealed everywhere, in Studio, in overviews and in blueprint data, and every read of it is written to the audit log. Blueprints can read both kinds, because the secure value is injected at execution time. Use secure values for API tokens, passwords and signing secrets such as postmark_token. **How do I configure email sending on a RUAL cluster?** Two keys: add your Postmark server token as the secure postmark_token setting, and set email_default_from to an address on your verified Postmark sender domain. From that point, password resets and any email-sending blocks work without further configuration. **How do I rotate an API token used by RUAL blueprints?** Store it as a custom secure key such as stripe_secret_key and read it at the start of the flow with get system setting. Rotating then means editing the value in one place, and every blueprint picks it up on the next run with zero blueprint edits. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Common Pitfalls · RUAL Documentation" description: "The mistakes that bite most often: caching, aggregations and silent no-ops." canonical: https://docs.rual.nl/blueprints/common-pitfalls language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Common Pitfalls Working with blueprints can be both invigorating and challenging simultaneously. In this section, we will highlight some of the most common pitfalls to be aware of when designing systems using RUAL. ### Not utilizing cache properly From our experience in designing high-scale, high-performance systems, it is crucial to differentiate when data needs to be real-time and when it can be effectively utilized from a cache. In many instances, it suffices to retrieve data that would typically be requested directly from a [storage](https://docs.rual.nl/blueprints/storage) system from a cached source. A common misconception is that the search process is so rapid that caching is unnecessary. However, the reason for its speed is precisely because we encourage caching. Failure to utilize caching may result in the storage system providing less accurate information unless explicitly specified for high precision. The utilization of a cache serves not only to enhance system performance but also to maintain that performance in high-scale operations. A single search query may not be a significant burden, but consider the scenario where the same query is executed 300 times within a minute due to 300 simultaneous page visits. In such cases, it would have been far more cost-effective to check if cached data is available, use it, and only request data from the storage when absolutely necessary. ## Real-time search results When performing a search query directed to a storage, the system does not wait for inserts to complete. Instead, it provides you with the most accurate data it currently has available. In some cases, this may result in a slight delay in the search data being updated. You might expect the newly created document to be included in the results, but it hasn't been incorporated into the search engine's data yet. If it is crucial for your system's functionality to have the latest and most up-to-date data, you can incorporate the [disable cache](https://docs.rual.nl/block-types/query/query_disabled_request_cache) option in your query. This instructs the core to wait until the latest data has been inserted before executing the search query. However, in storage systems with a high rate of inserts, enabling this option could potentially lead to significant delays in the execution of the search query. ## Inappropriate use of aggregations Utilizing aggregation queries to consolidate data is indeed a powerful feature, but it comes at a significant cost. Aggregations should be reserved for sparingly updated storages; employing them on heavily populated storages can lead to performance degradation and excessive memory usage. For efficient handling of aggregated data, it is advisable to store such information separately and retrieve it through standard queries. For example, when fetching aggregated data for users and you want to keep a `count` of their posts, it is more efficient to maintain a `count` attribute within the user object and increment it as necessary. Alternatively, you can leverage Redis cache to store aggregated statistics. This approach involves storing JSON with aggregated results from relevant queries, making counts instantly accessible in milliseconds. These counts can be updated periodically using a [time interval](https://docs.rual.nl/blueprints/repeating-events) or in response to [storage events](https://docs.rual.nl/blueprints/storage-events). ## Frequently asked **Why does my RUAL search not return a document I just created?** Searches do not wait for inserts to complete, so a just-created document may not be incorporated into the search engine's data yet. If your flow truly needs the latest data, add the disable cache option to the query, but expect noticeable delays on storages with a high insert rate. **Are aggregation queries slow in RUAL?** They come at a significant cost on heavily populated storages, where they degrade performance and use excessive memory. Reserve aggregations for sparingly updated storages, and for counts prefer maintaining a counter field you increment, or storing aggregated statistics in Redis updated on an interval or from storage events. **Do I need to cache search results in RUAL?** For read-heavy flows, yes. A single search query is cheap, but 300 simultaneous page visits run the same query 300 times within a minute. Checking a Redis cache first and only hitting storage when necessary keeps the system fast at scale. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Precompiling · RUAL Documentation" description: "What precompiling does to execution speed and when to reach for it." canonical: https://docs.rual.nl/blueprints/precompiled language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Precompiled Data When engaging with Blueprints, you may observe that certain data is labeled as 'fast'. This denotes precompiled data, which is designed to enhance the efficiency and speed of blueprints. ### Precompiling for Enhanced Performance To ensure blueprints operate at peak efficiency, we focus on determining the output values of blocks in advance. This involves pre-building functions and blocks, a process that saves execution time in the long run. We refer to this optimized data as `precompiled`, a term you'll encounter frequently in our documentation. Each time a blueprint is saved, a signal is sent to all production nodes to discard the existing precompiled data. Additionally, if a blueprint hasn't been executed on a node for at least 6 hours, the node will automatically clear its precompiled information to conserve memory. We recommend reusing the same block for identical values within a blueprint, as more connections are technically more cost-effective. When dealing with functions, it's beneficial to limit the use of custom object data. For instance, instead of passing an entire object to a function, modal, or queue item, only pass the necessary elements like the `guid` when that's all you need. ## Precompiled value In the example provided below, you can observe a precompiled value, marked by a green icon. This icon signifies that the value was precompiled during a previous execution and has been stored for efficient use in future operations. ## Frequently asked **What does the fast label on blueprint data mean in RUAL?** It marks precompiled data: RUAL pre-builds the output values of blocks in advance to save execution time. A green icon on a value signifies it was precompiled during a previous execution and stored for efficient use in future runs. **When does RUAL discard precompiled data?** Every time a blueprint is saved, a signal tells all production nodes to discard its existing precompiled data, and a node automatically clears precompiled information for blueprints it has not executed for at least 6 hours, to conserve memory. **How do I make my RUAL blueprint faster?** Reuse the same block for identical values instead of duplicating blocks, because more connections to one block are more cost-effective, and pass only the data a function, modal or queue item needs, for example just the guid instead of an entire object. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Remote access control · RUAL Documentation" description: "Scopes, the scopes modal, rate limits and throttles on your own endpoints." canonical: https://docs.rual.nl/blueprints/remote-access-control language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Remote Access Control Seamlessly manage access for all your pages and endpoints, including advanced methods for securing API endpoints, ensuring streamlined control and robust protection across your system. ### Basic Access Control Information Our system includes an advanced access control mechanism called **Scoping**. This feature allows you to assign specific scopes to users, functioning as digital keys to access certain pages and APIs. Scoping ensures secure and controlled entry management. For a detailed tutorial on assigning scopes to users, refer to [Giving Users a Scope](https://docs.rual.nl/cluster/user-access-management#giving-user-permission). When a state page is created without any assigned scopes, it becomes accessible to all users by default. This configuration is referred to as the `*public` scope, providing unrestricted access to the page. To easily determine whether a page is public or private, you can check the scope menu button's icon without opening the menu: **Open Lock Icon:** Indicates that the page is `public` and accessible to all users. - **Closed Lock Icon:** Indicates that the page is `private` and requires users to have the necessary scope for access. The image below illustrates both scenarios for better understanding. ### Scopes Management Modal To manage scopes for a specific block, click on the lock icon associated with it. This will open the **Scopes Management Modal**, where you can perform the following actions: - **Select Additional Scopes:** Choose existing scopes to grant users access to the block. - **Deselect Unnecessary Scopes:** Remove scopes that are no longer required for access to that state page or API. - **Create a New Scope:** If the required scope does not already exist, simply type the desired name into the search input. A green button will appear, allowing you to create the new scope instantly. ### Default Scopes When you create a new RUAL cluster, two default scopes are automatically generated: | Scope name | Description | | --- | --- | | `*public` | This scope serves as the default for all state pages and APIs. If no other scopes are assigned to a state page or API, they will automatically use the `*public` scope, making them accessible to everyone. | | `*loggedin` | This scope is optional and not selected by default. When applied, it restricts access to logged-in users only, ensuring that unauthenticated users cannot view the page. Instead, they will be redirected to the forbidden access page (`/403`). | ### API-Specific Access Control For API endpoints, in addition to the standard scoping system we have introduced two advanced access control options in [RUAL Core v13.0.7](https://docs.rual.nl/core-versions/130007): ### API Rate Limiting This feature allows you to add a rate limit to an API in just a few seconds. The rate limit settings modal includes an enable switch and three configurable fields: #### Type Select the key type used for managing the rate limit. There are five options available: - **URL:** Applies a rate limit to the entire API URL, covering all methods for all users. - **URL + User:** Applies a rate limit to the entire API URL, covering all methods but specific to each user. - **Method + URL:** Applies a rate limit to the combination of the API URL and the method, specific to each method but for all users. - **Method + URL + User:** Applies a rate limit to the combination of the API URL, the method, and the user, specific to each method and user. #### Max Requests in Timeframe Set the maximum number of requests the key can make within a specified timeframe using this input field. #### Timeframe in Seconds Define the duration of the timeframe in seconds using this input. For example, if the timeframe is set to 30 seconds and the max requests are set to 10, the key can make up to 10 requests within 30 seconds. Any additional requests during that timeframe will be blocked. ### API Throttle This feature enables you to add a throttle to an API in just a few seconds, effectively functioning as a lock-and-wait system. The throttle settings modal includes an enable switch and two configurable fields: #### Type Select the key type used for managing the throttle. There are five options available: - **URL:** Applies a throttle to the entire API URL, covering all methods for all users. - **URL + User:** Applies a throttle to the entire API URL, covering all methods but specific to each user. - **Method + URL:** Applies a throttle to the combination of the API URL and the method, specific to each method but for all users. - **Method + URL + User:** Applies a throttle to the combination of the API URL, the method, and the user, specific to each method and user. #### Timeout Use this input to set the timeout duration for API requests. For instance, if the timeout is set to 10 seconds, the key will be allowed to make one request every 10 seconds. ## Frequently asked **How do I make a RUAL page or API private?** Click the lock icon on the block to open the Scopes Management Modal and assign a scope. A state page without scopes uses the *public scope and is accessible to everyone, *loggedin restricts it to authenticated users, and you can create a custom scope by typing its name into the modal's search input. **How do I add a rate limit to a RUAL API endpoint?** Open the endpoint's rate limit settings, enable it, pick the key type (URL, URL + User, Method + URL, or Method + URL + User), and set the maximum requests within a timeframe in seconds. Additional requests within that timeframe are blocked. **What is the difference between a rate limit and a throttle in RUAL?** A rate limit blocks requests beyond a maximum within a timeframe, while a throttle works as a lock-and-wait system with a timeout, for example allowing one request every 10 seconds. Both are configured per endpoint with the same key-type options, and both were introduced in RUAL core v13.0.7. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Service Providers · RUAL Documentation" description: "Credentials for external services, held by the cluster so a key never sits on a pin." canonical: https://docs.rual.nl/blueprints/service-providers language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Service Providers Credentials for external services live on the cluster, not in your blueprint. Where a block supports it, you give it the name of a provider and it fetches what it needs at the moment of the call, uses it and lets it go, so the key never lands on a pin and never appears in a trace. Every integration has the same awkward moment. Your flow needs to call Stripe, so it needs the Stripe secret key. The obvious place to put it is on the block's input pin, which means the key is now part of the blueprint, which means everybody who can open that blueprint can read it. Simulation makes it worse rather than better: a trace hands back every value that flowed through the run, so the key is not just stored where developers can see it, it is printed to them each time they check their work. A service provider moves the credential off the canvas entirely. You configure it once on the cluster, and blueprints refer to it by a name you choose. The block resolves the credential at request time, uses it for that one call, and does not keep it. Nothing about the key reaches a pin, a trace, a log line or the console. ## What the blueprint carries A name. `stripe-live`, `postmark-main`, `notifications`. It is not sensitive, it is readable in the canvas, and it says plainly which account the flow is talking to. Two environments become two providers and one name change. ## What the cluster holds The key, the token, the signing secret, the webhook URL. Written once, never handed back: reading a provider shows every secret field as "set" rather than as its value, no matter who is asking or how they ask. ## Setting One Up Providers are managed in RUAL Studio, alongside the other cluster settings. Creating one takes four decisions. - **Pick the type.** The type is the service itself: Stripe, Postmark, Anthropic, MessageBird. It decides which fields you are asked for and which of those fields are secret. - **Give it a name.** This is what your blueprints will say. Name it after the account rather than the vendor when you have more than one: `stripe-live` and `stripe-test` beat `stripe` and `stripe2`. - **Fill in the fields.** Secret fields go in once. From then on the studio shows them as set, never as their value, and leaving them untouched on a later edit leaves them unchanged. - **Test it.** A provider starts untested, and testing it makes a real call against the service. A provider whose credentials you edit goes back to untested, because the previous result no longer describes what is stored. After that, the provider is either named on a block or reached for by the cluster on its own, depending on the service. The next section is the definitive list of which is which. ## A Provider Declares What It Can Be Used For Every provider type declares the purposes it can serve, and everything that resolves one asks for a specific purpose. That pairing is what makes a provider name safe to type: naming your mail provider on a payment block fails immediately and says why, instead of sending Stripe a Postmark token and leaving you to interpret whatever the remote service says about it. #### Providers a block names directly These blocks have a `provider` in-pin. Type the provider's name there and no credential exists anywhere in the blueprint. | Purpose | Provider type | Blocks | | --- | --- | --- | | Payments | Stripe | [`function_create_stripe`](https://docs.rual.nl/block-types/stripe/function_create_stripe), and through the connection it hands out, every checkout, payment, subscription, refund and credit note block | | Chat notifications | mssgs | [`mssgs_send_message`](https://docs.rual.nl/block-types/mssgs/mssgs_send_message), for posting an operational message into a channel from a flow | | Horticulture trade | Floriday | [`function_floriday_open_connection`](https://docs.rual.nl/block-types/floriday/function_floriday_open_connection) and its supplier variant | #### Providers the cluster uses on its own These are bound by purpose rather than named on a pin: exactly one provider per entity serves each, and the cluster reaches for it without any blueprint being involved. | Purpose | Provider types | What uses it | | --- | --- | --- | | SMS delivery | MessageBird, Twilio | Two-factor codes, passwordless login codes and phone enrolment | | Transactional email | Postmark, Lettermint | The cluster's own outbound mail | | Node reporting | mssgs | Where the node reports its own errors and warnings, deliberately a different provider from the one your flows post to | #### Provider types you can configure ahead of the blocks These types exist and can be filled in and tested, but nothing resolves them from a blueprint yet. Configuring one now is not wasted: it is where the credential belongs, and the block that reads it is the part still to come. Until then, those integrations still take their key on a pin or from a secure system setting. | Purpose | Provider types | | --- | --- | | AI completion | Anthropic | | Marketing email | Mailchimp | | Voice calls | Twilio, kept separate from SMS so one account can serve one and not the other | | Chat notifications | Discord | | Source control | GitHub | | Geo database updates | MaxMind | | Map tiles | Mapbox | | Cache and search connections | Redis, Elasticsearch | ## What the Cluster Guarantees When a Block Asks Handing a credential to a block is the moment where this could go wrong, so three rules are enforced on the block's behalf rather than left to it. | Rule | What it means for you | | --- | --- | | The credential is never put anywhere the run can read it | It does not become part of the connection object, an out-pin, the values a simulation hands back, or a console line. Blocks are written so the connection carries the provider's name and nothing else. | | A name only reaches your own providers | You can resolve your own, and the ones the hosting entity shares with you. A name is never a route into another tenant's account, however precisely you guess it. | | The wrong kind of provider is refused before the credential is fetched | Asking for a payment credential and naming a mail provider is an error with a clear code, raised locally. Nothing is sent anywhere and no key changes hands. | > **A webhook URL is a credential** An incoming webhook URL has its token built into the address, so a chat webhook sitting on a pin is a secret sitting on a pin in plain sight. Chat destinations are providers for exactly that reason, and the URL is stored as a secret field like any key. ## Rotating a Key Is One Edit Open the provider, paste the new value, save. Every flow picks it up on its next run: no blueprint is edited, nothing is redeployed, nothing is reactivated. The same applies to swapping accounts, because the flows named a provider rather than an account. The most common shape is one provider per environment. Point the test blueprint at `stripe-test` and the live one at `stripe-live`, and moving from one to the other is a single value on a single block rather than a key nobody wants to paste into a canvas twice. ## Compared With the Older Ways There were two patterns before this, and both still exist. It is worth being clear about when each is the right answer. | Where the value lives | Use it for | Not for | | --- | --- | --- | | **A service provider** | Every service in the first two tables above, and any service with a provider type you want the credential stored properly for today. It is the only option where the value cannot be read back out at all. | Values that are not credentials, and services with no provider type yet. | | **A secure [system setting](https://docs.rual.nl/blueprints/system-settings)** | Credentials for a service whose blocks do not read a provider yet, such as the [Anthropic key](https://docs.rual.nl/tutorials/ai-endpoints), and cluster-wide values a flow reads at runtime. | Anything a block can name a provider for. A setting is read by a block, which means the value enters the run and can be printed by the next block along. | | **A value on a pin** | Non-secret configuration: an account identifier, a sender address, a region, an API version. | Any secret at all. Whatever is on a pin is in the blueprint, readable by anyone who can open it and visible in every simulation of it. | > **Check which table a service is in before you go looking for a pin** Most integration blocks predate providers and still take their key on a pin. A provider type existing for a service does not yet mean a block reads it, so if you cannot find a `provider` pin, the first table is the definitive list of blocks that have one. ## When a Block Cannot Resolve a Provider | What you see | What it usually is | | --- | --- | | The provider is not found | The name on the block does not match the provider's name, or the provider belongs to a different entity. Names are exact. | | The provider is the wrong purpose | The named provider is a real provider of a type that cannot serve this block. Check that the payment block names the payment provider. | | The provider is disabled | It exists and is switched off. That is a deliberate state, so somebody turned it off on purpose. | | The remote service rejects the call | The credential resolved and the service refused it. Test the provider from the studio: that call goes straight to the service and separates a stale key from a wiring problem. | | SMS is not sent and nothing looks wrong | Outbound SMS resolves a provider first and falls back to a cluster setting. A cluster whose token only exists in a node configuration file has no credential the platform can see: move it into a provider. | ## Next Steps ## Frequently asked **Where should I store an API key in RUAL?** In a service provider, configured on the cluster, whenever the service has one. Blueprints then refer to it by a name you choose, and the block resolves the credential at the moment of the call, so the key never lands on a pin, never appears in a simulation trace and cannot be read back out. Blocks that do not read a provider yet should take the key from a secure system setting instead, never from a value in the canvas. **How do I rotate an API key in RUAL?** Open the service provider, paste the new value and save. Every flow picks it up on its next run. No blueprint is edited, nothing is redeployed and nothing is reactivated, because the flows named a provider rather than a key. **Why does my RUAL block say the provider is the wrong purpose?** Every provider type declares which purposes it can serve, and every block asks for a specific one. The named provider exists but is a type that cannot serve this block, for example a mail provider named on a payment block. The check happens before any credential is fetched, so nothing was sent anywhere. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Places and Geo Search · RUAL Documentation" description: "Search the built-in places dataset, filter by proximity and enrich your own documents with it." canonical: https://docs.rual.nl/blueprints/places language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Places and Geo Search Search the built-in places dataset, ask radius and map questions, filter your own documents by what is nearby, and enrich them with it. All offline, against data shipped with the node. The `places_*` and `geo_point_*` blocks answer location questions against datasets on the node itself: no API keys, no per-call cost, no request leaving the building. Lookups are fast enough to run inside loops and over whole inventories. This guide covers the search and filtering flow; [Location Blocks](https://docs.rual.nl/reference/location-blocks) lists every block with its pins, and [`places_dataset_info`](https://docs.rual.nl/block-types/places/places_dataset_info) tells you which dataset release your node serves and whether it is `loaded` at all. ## 1. Text Search [`places_search`](https://docs.rual.nl/block-types/places/places_search) (search places) is the one search box over both points of interest and settlements. It handles accents (`Bréda` finds `Breda`), typos (`amsterdm centraal`), and partial words as the user types, and collapses duplicate records from the source data. The outputs are a ranked `results` array, the `best` match and a `count`. - `near` plus `bias_radius_m` biases ranking toward a point without excluding anything farther away. - `limit`, `typo_tolerance` and `min_confidence` tune how much comes back and how forgiving the matching is. - `kinds` and `categories` restrict the search to points of interest, settlements, or specific category groups. A typed query, typos and all, goes into [search places](https://docs.rual.nl/block-types/places/places_search). A [from city](https://docs.rual.nl/block-types/geopoint/geo_point_from_city) point biases the ranking toward Breda, a [number](https://docs.rual.nl/block-types/number/number_default) caps the result count, and [get fields](https://docs.rual.nl/block-types/object/object_field_getter_multiple) reads the name off the best match for the reply. [![Studio canvas example for the search places block: one search box over every place.](https://docs.rual.nl/canvas-examples/places_search.png)](https://docs.rual.nl/canvas-examples/places_search.png?v=3) ## 2. Radius and Map Questions Three blocks answer "what is here", each for a different question: | Block | Answers | Notes | | --- | --- | --- | | [`places_nearest`](https://docs.rual.nl/block-types/places/places_nearest) (nearest places) | The closest points of interest to a geo point, ordered by distance, each with its distance in metres. | Leave `max_distance` empty for an unbounded search and apply your own cutoff later. | | [`places_within`](https://docs.rual.nl/block-types/places/places_within) (places within radius) | Every point of interest inside a radius, nearest first. | Compare `count` with `raw_count` to see how much duplication the source data carries at that spot; `truncated` flags a capped result. | | [`places_in_bbox`](https://docs.rual.nl/block-types/places/places_in_bbox) (places in bounding box) | Every point of interest inside a bounding box: the block for rendering what is currently on a map screen. | Results carry no distance (a box has no query point); use the nearest block when you need distances. | ## 3. Categories The source dataset ships roughly 1700 raw categories, far too many to show a user, so they are folded into curated groups with Dutch and English labels. [`places_categories`](https://docs.rual.nl/block-types/places/places_categories) (list place categories) lists those groups, the block to build a category dropdown from. [`places_resolve_category`](https://docs.rual.nl/block-types/places/places_resolve_category) (resolve place category) goes the other way: free text such as `kroeg` or `supermarkt` becomes a category group id, with a `confidence` score and `alternatives` when a word spans several groups. Plurals are handled, so `kroegen` and `bars` just work. ## 4. Typed Filters, From Sentence to Result [`places_parse_query`](https://docs.rual.nl/block-types/places/places_parse_query) (parse places query) turns typed Dutch or English into a structured filter: `binnen 300m van een basisschool` becomes a proximity filter, `niet bij scholen` a negated one, `minimaal 3 kroegen binnen 500m` carries a minimum count. Two blocks consume that filter: - [`places_match`](https://docs.rual.nl/block-types/places/places_match) (match places filter) tests one geo point against the filter. Its `distance_m` and `count` come back even when nothing matched, so you can re-answer a different radius without another call. - [`places_filter`](https://docs.rual.nl/block-types/places/places_filter) (filter by places) keeps or removes items of a whole list by the filter, reading each item's point from `point_path`. With `annotate` it writes the distance onto each item instead of removing anything. ## 5. Enrich and Facet Your Own Documents [`places_enrich`](https://docs.rual.nl/block-types/places/places_enrich) (enrich with nearby places) adds the nearest place per category to every item in a list in one call, no iterator needed. The usual pipeline: query your documents with [`function_search`](https://docs.rual.nl/block-types/storage/function_search), run the enrich block, then write the result back with [`mutations_set_bp_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_set_bp_field_multiple) and [`function_update_document_mutations`](https://docs.rual.nl/block-types/storage/function_update_document_mutations). After that an ordinary range query filters on the stored distances, and the expensive proximity work happens once per document instead of once per page view. [`places_facet`](https://docs.rual.nl/block-types/places/places_facet) (facet counts by places) answers "412 properties near a hospital" in one block: it counts how many items of a list are near each category. An item with three hospitals nearby still counts once, which is what a facet over your own inventory means. [`places_density`](https://docs.rual.nl/block-types/places/places_density) (place density) profiles a single spot instead: totals, per km², and a classification. ## 6. Geo Point Helpers Every places block takes or returns a `geo_point`. The helpers create, check and resolve them: - [`geo_point_lat_lng_to_point`](https://docs.rual.nl/block-types/geopoint/geo_point_lat_lng_to_point) builds a point from two numbers; [`geo_point_to_lat_lng`](https://docs.rual.nl/block-types/geopoint/geo_point_to_lat_lng) splits one back out. - [`geo_point_from_city`](https://docs.rual.nl/block-types/geopoint/geo_point_from_city) resolves a city name to its centre point plus radius, optionally constrained by country code and province. - [`geo_point_reverse_geocode`](https://docs.rual.nl/block-types/geopoint/geo_point_reverse_geocode) resolves a point to the nearest street, city, province and country. - [`geo_point_snap`](https://docs.rual.nl/block-types/geopoint/geo_point_snap) pins a point onto the nearest road or railway and reports how far it moved; [`geo_point_nearby`](https://docs.rual.nl/block-types/geopoint/geo_point_nearby) checks whether such a network is within a given distance. - [`geo_point_validate`](https://docs.rual.nl/block-types/geopoint/geo_point_validate) checks a point holds a valid latitude and longitude before you trust it. ## Notes From Practice - Store a place id when you want to refer to it later and re-resolve it with [`places_get`](https://docs.rual.nl/block-types/places/places_get) (get place by id). Branch on its `found` pin: an id can disappear between dataset releases, and not finding it is a successful lookup, not an error. - Results carry a confidence; raising `min_confidence` trades recall for precision on messy source data. - All of this runs offline, so there is no rate limit to design around. The cost is dataset freshness: check the release with the dataset info block when results look stale. - [Location Blocks](https://docs.rual.nl/reference/location-blocks): Every geo point, places and map block with its pins, in one lookup page. - [Storages](https://docs.rual.nl/blueprints/storage): The queries and mutations behind the enrich-and-write-back pipeline. - [Common Blueprint Patterns](https://docs.rual.nl/blueprints/common-patterns): The endpoint and error-handling shapes to wrap around a places search. ## Frequently asked **Does the RUAL places search need an API key?** No. The places blocks run offline against an Overture dataset shipped with the node, so there is no API key, no rate limit and no per-call cost, and lookups are fast enough to run inside loops. **How do I search places by typed text in RUAL?** Use places_search. One search box covers points of interest and settlements, tolerates typos and accents, and can bias results toward a geo point without excluding farther matches. The results array, the best match and a count come back ranked. **How do I filter my own documents by nearby places in RUAL?** Turn a typed sentence into a filter with places_parse_query, then apply it with places_filter to keep or remove items of a list, or places_match to test a single point. To make the result queryable later, run places_enrich once and write the nearest-place distances back onto the documents. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Interfaces · RUAL Documentation" description: "Building the front end: RUAL Studio, the component libraries, and how to override the pages that ship by default." canonical: https://docs.rual.nl/interfaces language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Interfaces Building the front end: RUAL Studio, the component libraries, and how to override the pages that ship by default. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Context Menu · RUAL Documentation" description: "The right-click menu on the canvas and the actions it can reach." canonical: https://docs.rual.nl/interfaces/contextmenu language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Context Menu Our frontend includes a default Context Menu built-in, designed to leverage available information on the page to accelerate your development process in RUAL. ## Modify Columns Where supported, columns in tables can be modified simply by right-clicking on them. The column that is currently clicked on will be highlighted in blue. When you select the column from the context menu, it will become hidden. These modifications are stored in your `localStorage`, linked to the specific `action_guid`. ## Find a Blueprint Block Finding blueprint blocks within the State UI is made easy by right-clicking on them. For every component we render, we track its corresponding blueprint block and make this information available to the frontend. Clicking on this option will open a new page, directing you straight to the relevant blueprint and block. ## Frequently asked **How do I hide a table column in RUAL?** Right-click the column where supported and select it from the context menu to hide it. Your modifications are stored in localStorage, linked to the specific action_guid of the table. **How do I find the blueprint block behind an element on a RUAL page?** Right-click the element in the State UI. For every component rendered, RUAL tracks its corresponding blueprint block, and choosing that option opens a new page straight to the relevant blueprint and block. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Components · RUAL Documentation" description: "How interface components are defined, configured and rendered." canonical: https://docs.rual.nl/interfaces/components language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Components When leveraging RUAL for frontend development, you are likely to encounter complex logic. To simplify frontend development, we are making full React functionality available to our RUAL developers in any cluster, starting with version 13.0.6 and subsequent versions. ## Existing Components By default, we provide [an extensive list of components](https://docs.rual.nl/block-types/state%20ui) within the blueprinting environment. These default blocks are React-based functional components, designed to equip the RUAL developer with a broad range of functionality integrated directly into [blueprinting](https://docs.rual.nl/blueprints/introduction). Although we aim to deliver most of the necessary functionality out of the box, we recognize that there may be unique needs that are not anticipated or covered by our default offerings. To accommodate the development of all types of frontend components, we offer full access to custom React functional component capabilities to RUAL developers through Custom Components in [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio). ## Custom Components Since [v13.0.6](https://docs.rual.nl/core-versions/130006) of RUAL Core, you will find an option titled "Manage Components" in the left-side menu. This section provides a comprehensive overview of the custom React components available across your entire cluster. These React components should be constructed as [React Functional Components](https://react.dev/learn/your-first-component) and are capable of supporting all the functionalities expected of any React component. There are minor modifications required to facilitate integration with blueprinting, which will be discussed in further detail below. ## Blueprint-Specific Parameters For blocks that support storages or fields, accessing these through parameters is necessary. This approach promotes component reuse, although these parameters do not automatically appear in your code. Below is a list of parameters you can use, along with their respective usage descriptions. | Parameter | Description | | --- | --- | | `bp_field` | A string that identifies the selected field name, which may be prefixed by the form identifier if a `form_id` inpin is present. | | `bp_multiple_fields` | An array of objects, each containing a selected field. | | `bp_hash` | A hash provided to FetchJS for retrieving fresh JavaScript when executing a script. | | `bp_img_src` | A URL or `data:` URL for an asset generated by the backend. | | `bp_submit_data` | An JWT-encoded object for submitting data to the backend. | | `bp_hyperlink` | The URL of the selected State Page. | | `bp_translate_key` | The translation key selected for the block. | | `bp_action_guid` | An identifier used by some blocks to execute queries and interact with the backend storage. | | `bp_iterate_component` | A path to the static custom component. Supply this to `RUAL.fetchComponent`, and it will return a valid cached React Component for your use. This is primarily utilized in iterative contexts, such as Advanced Datatables. | Certain inpin IDs are not directly usable within a React Component. Below, we detail how these are remapped. | Inpin ID | Parameter | | --- | --- | | `id` | `rual_id` | ## Trigger a Blueprint Function The example below illustrates how to trigger a blueprint function. Initially, you must define an inpin. In this case, we've used an inpin named `onclick` of the type `Executable Function` with the identifier `onclick`. ``` const { runBlueprintFunction } = window.RUAL; export default function CustomButtonComponent ({ uiclass, uistyle, uiattributes, onclick }) { function triggerFunction () { runBlueprintFunction(onclick, { 'value': 'custom-value' }); } return ( ); } ``` ## Using React Hooks Here is an example of a custom component that leverages React State. You can import any standard [React hook](https://react.dev/reference/react/hooks) for use in such components. ``` const { useState } = window.React; export default function CustomButtonComponent ({ uiclass, uistyle, uiattributes }) { const [isLoading, setIsLoading] = useState(false); function triggerFunction () { setIsLoading(true); setTimeout(() => { setIsLoading(false); }, 1000); } return ( ); } ``` ## Support for React Native `className` If you're familiar with React Native, you might be aware that React Web does not accommodate object and array-based `className` properties. However, in RUAL, we provide support for these formats, enabling you to effortlessly combine and utilize multiple classes from inpins or other properties. ``` // uiclass = { 'bpclass': true } export default function CustomClassNameComponent ({ uiclass }) { const classArrayExample = ['card-body']; return (
Class will be card form-control bpclass card-body
); } ``` ## Frequently asked **Can I write my own React components in RUAL?** Yes. Since RUAL Core 13.0.6, the Manage Components section in RUAL Studio lets you register custom React functional components on the cluster. They support standard React features such as hooks, with minor modifications to integrate with blueprinting. **How does a custom RUAL component receive the field selected on the blueprint?** Through parameters such as bp_field, a string holding the selected field name, and bp_multiple_fields for multiple selections. These parameters promote component reuse, but they do not automatically appear in your code. **How do I trigger a blueprint function from a custom RUAL component?** Define an inpin of the type Executable Function, for example one named onclick, and call it from your component code. The blueprint then runs the flow wired to that pin. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "RUAL Library · RUAL Documentation" description: "The helper library available inside custom components." canonical: https://docs.rual.nl/interfaces/rual-library language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # RUAL Library Custom Components have the ability to request specific JavaScript functions from the RUAL library. This page is dedicated to documenting the functions that are directly accessible, along with scenarios in which they might prove beneficial. ## How to Use Accessing RUAL Library functions is straightforward using `window.RUAL`. You can then spread the functions from the Library into your component. See the example provided below for guidance. ``` const { runBlueprintFunction } = window.RUAL; export default function CustomButtonComponent ({ onclick }) { function triggerFunction () { runBlueprintFunction(onclick, { 'value': 'custom-value' }); } return ( ); } ``` ## List of Functions Below is a list of functions available from the RUAL library for use within any [Custom Component](l/interfaces/components): | Call | Params | Description | | --- | --- | --- | | `globalState` | Class | This function exposes the `globalState` along with its functions. Below is a list of functions related to `globalState`. | | `globalState.set` | `key`, `newValue` | Sets a new value for the specified global key, updating the backend and refreshing the relevant pins and components. | | `useGlobalState` | `defaultValue`, `globalKey`, `uid` | Enables the use of a global state key within your Custom Component. The return value functions similarly to React's `useState`. | | `translate` | `key` | Attempts to translate the specified key using the currently active language. | | `fetchJS` | `uid`, `hash` | Retrieves fresh JavaScript data from the backend using the supplied `hash`. This function is typically used internally. | | `ws` | Class | Exposes the WebSocket and its functions. Below is a list of functions related to `ws`. | | `ws.request` | `method`, `uri`, `options` as `{ body, query }` | Performs the specified request to the backend, enabling API execution within the Custom Component. Returns a `Promise`. | | `settings` | Object | Provides globally available RUAL settings, such as timezone and other relevant rendering information. | | `fetchComponent` | `componentFile` | Returns a valid cached React Component for use, given a path to the static custom component. This is mainly used in iterative contexts like Advanced Datatables. | | `utils.getDeepvalue` | `data`, `key`, `defaultValue` | Accesses the specified `key` within an object and returns the `defaultValue` if it is not found. | ## Frequently asked **How do I call RUAL functions from a custom React component?** Through window.RUAL. You spread the functions from the RUAL library into your component and call them directly, for example translate, useGlobalState or fetchComponent. **How do I make an API request from a RUAL custom component?** Use ws.request with the method, the uri and an options object holding body and query. It performs the request to the backend from inside the custom component and returns a Promise. **How do I use a global state key inside a RUAL custom component?** Call useGlobalState with a default value, the global key and a uid. The return value works like React's useState, and globalState.set writes a new value that updates the backend and refreshes the relevant pins and components. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "RUAL Components · RUAL Documentation" description: "The component set that ships with RUAL and the props each one takes." canonical: https://docs.rual.nl/interfaces/rual-components language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # RUAL Components Building custom components in RUAL Studio offers an excellent opportunity to maximize the potential of React and RUAL together. However, there may be instances where you wish to integrate existing components within RUAL alongside your own creations. ## Our Default Components By default, we offer [an extensive selection of components](https://docs.rual.nl/block-types/state%20ui) within blueprinting. These standard blocks are React-based functional components, tailored to provide the RUAL developer with a wide array of functionalities seamlessly integrated into [blueprinting](https://docs.rual.nl/blueprints/introduction). On this page, we will discuss and demonstrate how you can incorporate these existing components within your custom components. Additionally, we will present a list of commonly used components along with their names. ## `UID` in components Most, reactive, components need their own unique identifier. This way, they can listen for changes in the global state or handle other RUAL based events. When injecting RUAL Components into your Custom Components, you'll need to extract the `uid` from the params. You can easily do this by prefixen the params with `uid,`. You'll need to make sure that each `uid` for each RUAL Component is unique, you can do this by prefixen them with an unique identifier in combination with the existing uid. ## Using Translations A frequently requested feature, which we also employ in our own blocks, is the translation component. This component enables you to request translations and incorporate HTML content into them. Additionally, we provide access to the [translate](https://docs.rual.nl/interfaces/rual-library#list-of-functions) function for integration within JavaScript/React. ``` import { Translate } from './translate_key.jsx'; export default function CustomExampleTranslate (props) { return (
); } ``` ## Using Inputs Our input components are seamlessly integrated with the global state through the use of state get/set key blocks in blueprinting. While constructing your own inputs is certainly feasible, it's often unnecessary. Below is an example showcasing how to utilize our pre-existing input component. ``` import { TextInput } from './input.jsx'; export default function CustomExampleInput ({ uid, onenter }) { return (
); } ``` ## List of Commonly Used Components Below is a compilation of RUAL Components that are commonly utilized within Custom React Components. Nearly all of these components accommodate the default parameters: `uiclass`, `uistyle`, and `uiattributes`. Regarding the `children` parameter, it indicates that the component supports React Children logic, which means other components should be nested within the tags of this component. The full props contract lives in the [Component Props Reference](https://docs.rual.nl/reference/component-props-reference). | Category | filename | params | | --- | --- | --- | | Inputs | `input.jsx` | uid, bp_field, type, onenter | | `input_file.jsx` | uid, bp_action_guid, bp_field, type, onenter | | | `input_price.jsx` | uid, bp_field, disabled | | | `input_date.jsx` | uid, bp_field | | | `input_checkbox.jsx` | uid, bp_field, value | | | Buttons | `button.jsx` | value, icon, iconclass, iconstyle, type, inpinfunction | | `button_content.jsx` | children, icon, iconclass, iconstyle, type, inpinfunction | | | Content | `translate_key.jsx` | bp_translate_key, language | ## Styling Components Accept the `uiclass`, `uistyle`, and `uiattributes` props and apply them to your component's root element. That is the contract that lets blueprint authors style instances from the canvas without touching your code. Keep everything inside the root on your own classes. The full contract, per-component props, and data-fetching patterns are in the [Component Props Reference](https://docs.rual.nl/reference/component-props-reference) and [RUAL Library](https://docs.rual.nl/interfaces/rual-library). ## Frequently asked **Can I reuse RUAL's built-in components inside my custom component?** Yes. RUAL's default components are React functional components, and you can embed them inside your own custom components. Commonly used ones include input.jsx, button.jsx and translate_key.jsx, and each accepts documented parameters. **Why does each RUAL component need a uid?** Most reactive components need their own unique identifier so they can listen for changes in the global state or handle other RUAL events. When you inject built-in components into a custom component, extract the uid from the params and prefix it with your own identifier to keep it unique. **How do I let blueprint authors style my custom RUAL component?** Accept the uiclass, uistyle and uiattributes props and apply them to your component's root element. That is the contract that lets blueprint authors style instances from the canvas without touching your code. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "RUAL Studio · RUAL Documentation" description: "The cluster management interface: where blueprints, storages and users are managed." canonical: https://docs.rual.nl/interfaces/rual-studio language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # RUAL Studio RUAL Studio is the visual flow editor for your cluster: the web application where you build blueprints, manage users and storages, and watch cluster traffic in real time. This page is a tour of the whole application. You work in RUAL Studio at [rual.at](https://rual.at). One installation serves every cluster you have access to: you pick a cluster, sign in, and Studio connects you to its blueprints, data, and settings. Everything you see in Studio lives on the cluster; Studio itself is only the editor and dashboard on top of it. ## Signing In Enter the hostname of your cluster and choose `Sign In`. Studio hands you over to the login page of that cluster, because credentials are always checked by the cluster itself, never by Studio. Clusters you signed in to before are remembered in the list, so returning is one click; remove an entry with the `x` next to it when you no longer need it. If you do not have an account yet, an administrator of the cluster creates one for you. [Getting Access](https://docs.rual.nl/cluster/getting-access) walks through receiving credentials and your first login. ## Finding Your Way Around The sidebar groups everything Studio can do into seven areas, and the `Search menu...` box at the top filters it instantly. The sections below walk through every group. Each entry links to the page that documents the underlying concept in depth, so you can treat this page as the map of Studio and the rest of the documentation as the territory. ## Dashboard The Dashboard is the landing area after sign in, built for answering one question quickly: is the cluster healthy, and what changed? - **General Information**: live HTTP traffic chart (requests, successes, errors), the recent blueprint activities (who created, renamed, or removed which blueprint and when), and the developers currently active on the cluster. - **Cluster Usage**: the numbers billing and capacity planning care about: automations run this month, traffic in GB, database and file storage, with graphs over 7, 30, or 90 days and year to date. - **Node Usage** and **Memory Usage**: per-node and per-worker resource utilization: CPU per worker, memory per node, node versions, types, and uptime. A cluster can run multiple nodes; these pages show each one separately. - **Audit Logs**: sign-ins with full access, privilege changes, deletions, and blocked outbound requests. Repeats are counted per hour, so `x4` means it happened four times. Only cluster owners can open this page; everyone else sees a permission notice. ## Blueprints The Blueprints group is where you spend most of your time: the editor itself, plus the shared resources blueprints rely on. #### Overview The overview lists every blueprint on the cluster with its active state and last modification. Each row shows who contributed to the blueprint and for what percentage, so you can see at a glance who to ask about a flow. Tags appear as chips on the row. The `Columns` button reconfigures the table, and `Recently Removed` lets you restore a blueprint that was deleted by accident. #### Creating a Blueprint `Create Blueprint` starts with a name and an optional namespace. Namespaces prefix the public functions of a blueprint, which keeps function names unique when blueprints call each other; [Best Practices](https://docs.rual.nl/blueprints/best-practices) explains when to use one. The next step asks what type of blueprint you would like. The six starting points are **Build a UI Page**, **Build a UI Modal**, **Build a Function**, **Build an API endpoint**, **Build an API Form**, and **Database event**. Each type pre-arranges the canvas with the starting blocks for that job, so you begin with a working skeleton instead of an empty canvas. #### The Canvas On the canvas you place blocks and connect their pins into flows: drag from an out-pin to an in-pin to connect, or drag into empty space to create and connect a new block in one motion. Colored blocks are starters and functions, the small circles are data and flow pins, and the lines between them are the order and data of execution. [Quickstart](https://docs.rual.nl/getting-started/quickstart) builds your first flow step by step, [Introduction to Blueprints](https://docs.rual.nl/blueprints/introduction) explains the mental model, and [All blocks](https://docs.rual.nl/block-types) documents every block you can place. The top bar holds the blueprint's lifecycle actions. Saving stores your changes in development, visible only to developers. `Activate` turns the blueprint on or off for triggers. Deploying pushes the saved state to production; the `Deployment` menu has a `compare` button to review the difference first. Saving is not deploying: read [How to Deploy](https://docs.rual.nl/deployment/how-to-deploy) before your first release. #### Manage Tags and Fields **Manage Tags** lists the labels you can stick on blueprints to group them in the overview. **Manage Fields** lists every field in the cluster with its type and parent: fields are the reusable building blocks of your data model, shared between storages, forms, and APIs. Deleting a field here removes it everywhere, so the list doubles as an impact check. #### Manage Storages, Redis Keys, and Settings **Manage Storages** shows every storage with its document count and creation date. Storages are the document databases your flows read and write; [Storages](https://docs.rual.nl/blueprints/storage) covers them in depth. **Manage Redis Keys** inspects the cluster's Redis cache directly, useful when debugging cached values; [Redis Keys](https://docs.rual.nl/cluster/api/redis-keys) lists the API equivalent. **Manage Settings** edits the system key-value settings of the cluster, from default language to date formats; [System Settings](https://docs.rual.nl/blueprints/system-settings) documents what each key does. #### Block Search, Disconnected Flows, Deprecated Actions Three maintenance tools earn their place as soon as a cluster grows: - **Block Search**: find blueprint actions by guid, event, function name, or storage, for example every block that writes to a storage you want to rename. - **Disconnected Flows**: actions whose flow pins are no longer wired to a starter, so they can never run. The list shows the block type, what is missing, and the blueprint it lives in: safe cleanup candidates. - **Deprecated Actions**: blocks that were deprecated by a core upgrade, so you can plan their replacement. [Version Management](https://docs.rual.nl/architecture/version-guide) explains the upgrade routine. ## Devices Devices pairs local hardware with the cluster. **IP Ranges** defines the network ranges devices may connect from, **Label Printers** registers printers the cluster can send labels to, and **Cobots** manages collaborative robots controlled through blueprints. Most clusters never touch this group; it exists for on-premise and industrial setups. ## User Interfaces Everything that shapes what end users see in the browser: - **Manage Translations**: the translation keys of the cluster with a flag per language. Blocks that render text pick a key here, and the visitor's language decides which value is shown. - **Manage Templates**: page templates with edit settings and revisions. Templates define the shared shell (menu, footer, scripts) around your pages; [Block Templates](https://docs.rual.nl/block-types/block-templates) explains the concept. - **Manage Assets**: the file storage for images, documents, and videos, with visibility and type filters and direct upload. [Assets](https://docs.rual.nl/blueprints/assets) shows how flows generate and serve them. - **Manage Components**: custom React components registered on the cluster, with their own code editor. [Components](https://docs.rual.nl/interfaces/components) documents how to build one; note the warning Studio shows: custom components trade blueprint readability for flexibility, so document their usage well. - **View Pages** and **View Modals**: read-only overviews of the state pages and modals currently defined on the cluster, across all blueprints. ## Cluster Traffic The traffic group answers who is calling the cluster, from where, and how it behaves: - **API History**: every incoming request with method, URL, status, duration, IP, and time, searchable by any field and filterable by period. Open a row for the full request and response. [API History](https://docs.rual.nl/cluster/api/api-history) covers the API side. - **Traffic Insights**: the aggregate view: totals, failures, average and slowest durations, an hourly traffic chart, and the most-used endpoints with their sparklines. - **Country Map**: a world heatmap of where requests originate, with the top countries listed. - **Banned IPs**: addresses the cluster blocks, managed manually or by automated abuse protection. ## Cluster Management - **Manage Domains**: the domains routed to this cluster, with the templates used for login, security, and password reset pages per domain. This is where you point a custom domain at your cluster. - **View Events**: the storage events defined on the cluster: flows that fire when documents change. [Storage Events](https://docs.rual.nl/blueprints/storage-events) explains the mechanism. - **View APIs**: every custom API endpoint registered on the cluster with its method, URL, security scopes, and status. - **View queue**: the executing and scheduled queue items. Long-running and delayed work lands here; [Queue](https://docs.rual.nl/blueprints/queue) documents how flows use it. - **All Logs** and **Live Logs**: the cluster's system logs, historical or tailed in real time with debug scopes. [Debugging](https://docs.rual.nl/troubleshooting/debugging) shows the live-log workflow. ## Cluster Settings The settings group controls who can sign in and what they are allowed to do: - **Users**: every account with its role, scopes, and last activity. Each row has quick actions: `Activity` opens that user's activity page, `Reset` clears the failed login counter to unlock an account, and the trash icon removes the account. [User Access Management](https://docs.rual.nl/cluster/user-access-management) walks through creating and editing users. - **Authentication Groups**: groups that restrict sign-in itself: require two-factor authentication, set expiry or inactivity rules, or limit access to whitelisted IPs. - **User Groups**: bundles of scopes assigned to users at once. Users in multiple groups receive the union of all scopes. [User Roles Explained](https://docs.rual.nl/cluster/user-roles-explained) helps you design them. - **User Activity**: who was active when and for how long, as a summary per user and as detailed records over a selectable period. - **Block Defaults**: the system defaults blocks fall back to when optional pins are not provided, starting with the language settings of the cluster. - **Node Settings** and **Node Updates**: per-node configuration and the rollout of core updates across nodes. [Version Management](https://docs.rual.nl/architecture/version-guide) covers safe upgrades. ## Shortcuts and Context Menus Studio is built for keyboard and right-click driven work. Right-clicking opens context menus almost everywhere: on the canvas, on blocks, on pins, and on table columns. Dragging from a pin creates and connects a new block in one motion. [Context Menu](https://docs.rual.nl/interfaces/contextmenu) documents the built-in menu, and [Tips & tricks](https://docs.rual.nl/blueprints/tips-and-tricks) collects the canvas shortcuts. ## Where to Go Next - [Quickstart](https://docs.rual.nl/getting-started/quickstart): from cluster login to a live page and API endpoint in 15 minutes. - [Core Concepts](https://docs.rual.nl/getting-started/core-concepts): blueprints, blocks, pins, flows, storage, and deployment. - [API Guide](https://docs.rual.nl/cluster/api-guide): everything Studio does is backed by cluster APIs you can call directly. - [Debugging](https://docs.rual.nl/troubleshooting/debugging): live logs, play mode, and inspection tools when a flow misbehaves. ## Frequently asked **Where do I open RUAL Studio?** You work in RUAL Studio at rual.at. One installation serves every cluster you have access to: you enter the cluster hostname, and Studio hands you to that cluster's own login page, because credentials are always checked by the cluster itself, never by Studio. **How do I restore a deleted blueprint in RUAL Studio?** The Blueprints overview has a Recently Removed option that lets you restore a blueprint that was deleted by accident. The overview also shows who contributed to each blueprint and for what percentage, so you know who to ask about a flow. **Where can I see who is calling my RUAL cluster?** In the Cluster Traffic group. API History lists every incoming request with method, URL, status, duration, IP and time, Traffic Insights aggregates totals, failures and the most-used endpoints, and the Country Map shows where requests originate. **Who can see the Audit Logs in RUAL Studio?** Only cluster owners. The Audit Logs page records sign-ins with full access, privilege changes, deletions and blocked outbound requests, and everyone else sees a permission notice instead. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Handle Iterations · RUAL Documentation" description: "Render lists and repeat structures without fighting the renderer." canonical: https://docs.rual.nl/interfaces/iterations language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Iterations Working with iterative, repeating content poses a significant challenge, particularly because we prioritize exceptionally fast rendering. Let's explore in depth how we maintain rapid performance with iterative content. ## When Iterative Components Are Used Our system is designed to render precisely what you request. However, in certain scenarios, such as with [Advanced Datatables](https://docs.rual.nl/block-types/state%20ui/state_advanced_datatable), iterative content becomes necessary. By default, we supply content directly from [storage](https://docs.rual.nl/blueprints/storage), but sometimes this approach does not suffice. Take, for instance, the need to display a status label within the datatable. One could utilize the `Row Map` function for this purpose, returning a `state` pin with the corresponding name. However, this method has a significant drawback: it necessitates rebuilding the entire component for each row from the backend and then fetching it on the frontend. In such situations, using iterate states offers a more efficient solution. ## How It Works When you create, for instance, an [Advanced Datatable](https://docs.rual.nl/block-types/state%20ui/state_advanced_datatable), an outpin named `Row Component` becomes available. Dragging from this outpin leads to the creation of the `execute component` block. Within this block, you can assign a [Custom Component](https://docs.rual.nl/interfaces/components) that the system will utilize for each row's individual column, reusing this component on the frontend to significantly enhance rendering speed. Additionally, backend-exclusive information can still be provided via the `Map Row` functionality, with the returned information from these being accessible to your component as well. ``` const { translate } = window.RUAL; export default function CustomComponent({ uid, index, columns, row }) { return ( {row.username} {row.firstname} {row.lastname} {row.email} {translate(row.language)} ); } ``` ## Exposed Iterate Parameters When using a Custom Component within iterators, the following parameters are automatically provided: | parameter | example usage | | --- | --- | | `uid` | Unique identifier for the row | | `index` | Position of the item (starting from 0) | | `columns` | Array of columns displayed in datatable | | `row` | row._meta.guid, row.username, row.firstname, row.lastname, row.email, row.language | ## Using a TR as the first item in the component To make this work, please use a [`tr`](https://docs.rual.nl/block-types/state%20ui/state_tr) element as the first item in the row component. This ensures the component fits properly in the datatable. ## Enabling JSON Right Click To enable the JSON right-click modal for a row, you must add `guid={row._meta.guid}` as a parameter to the [`tr`](https://docs.rual.nl/block-types/state%20ui/state_tr) element in the row render component. ## An Example Below is a straightforward example demonstrating how to use iterate state components. Notably, in React, it's possible to return data without enclosing it within an HTML tag. This approach is highly recommended as fewer DOM nodes lead to faster rendering. ``` const { translate } = window.RUAL; export default function CustomComponent({ uid, index, columns, row }) { return ( {row.username} {row.firstname} {row.lastname} {row.email} {translate(row.language)} ); } ``` ## Frequently asked **How do I render a custom component per row in a RUAL datatable?** Drag from the Row Component outpin of the Advanced Datatable to create an execute component block and assign your custom component to it. The frontend reuses that component for each row, which is much faster than returning a state through Row Map, because that rebuilds the entire component per row on the backend. **What data does a RUAL row component receive?** The iterate parameters uid, index, columns and row are provided automatically, and anything returned from the Map Row functionality is accessible to your component as well. **Why should a RUAL datatable row component start with a tr element?** So the component fits properly in the datatable. Add guid={row._meta.guid} to the tr element when you also want the JSON right-click modal enabled for the row. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Overwrite Defaults · RUAL Documentation" description: "Replace the built-in login and system pages with your own." canonical: https://docs.rual.nl/interfaces/overwrite-default-pages language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Overwrite Default Pages Sometimes the default settings may not meet your needs. In RUAL, you have the option to override our defaults using State Pages. You can redefine and customize pages simply by using State Pages to specify new definitions. ## How it works You can overwrite default pages by using a [State Page](https://docs.rual.nl/block-types/state%20ui/state_page). This function will request the URL you wish to use for this page. You can specify any default page, such as `login`, to overwrite it. Once you activate and deploy this page, the default login will be replaced by the page you have created. You can also limit this strictly to development by appending the `?development` query to your URL when logged in as a RUAL Development User. Repeat this process for any default page listed that you wish to overwrite, and ensure it is deployed to make the page available to all public users within your cluster. ## Pages allowing overwrite The pages listed below are provided by default with their standard logic, but can be overwritten using State Page blocks. | Page | Description | | --- | --- | | `login` | The default login page, utilized by users to gain access when attempting to reach restricted or authentication-required areas. | | `passwordreset` | The default password reset page, available to users wishing to reset their passwords. | | `security` | The default security page, used by users to view and manage their active sessions and multi-factor authentication options. | ## Login Tips When developing your own login system, you are responsible for implementing comprehensive security measures. Fortunately, RUAL addresses several basic security concerns by default, such as form hoisting and input sanitization, to help facilitate this process. We continue to enforce essential security measures, such as preventing mass multi-authentication attempts from the same IP address and blocking repeated unsuccessful authentication attempts to the same user account. If your login flow fails and you find yourself locked out of the cluster, you can append `?studio=request` to the URL to regain access to the default login page. Any non-empty value works, and the built-in page renders even when your custom page is broken, blank or deactivated. [Login Escape Hatch](https://docs.rual.nl/interfaces/login-escape-hatch) covers this recovery path in full. ## Frequently asked **Which default RUAL pages can I overwrite?** The login, passwordreset and security pages. Build a State Page that requests the URL of the page you want to replace, activate it and deploy, and from then on your page is served instead of the built-in one. **How do I test an overwritten default page without affecting users in RUAL?** Append ?development to the URL while logged in as a RUAL Development User. The overwrite then applies strictly to development until you deploy the page. **What security does RUAL still enforce on a custom login page?** RUAL keeps preventing mass multi-authentication attempts from the same IP address and blocks repeated unsuccessful authentication attempts against the same user account, and it handles form hoisting and input sanitization. The broader security of your login flow is your own responsibility. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Login Escape Hatch · RUAL Documentation" description: "Force the built-in login page with ?studio=1 when a custom login page locks you out." canonical: https://docs.rual.nl/interfaces/login-escape-hatch language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Login Escape Hatch A custom login page that breaks would normally lock every developer out of the cluster it belongs to. It cannot: any non-empty studio query parameter bypasses user-defined page resolution and forces the built-in login page to render. > **Locked out right now?** Open `https://your-cluster/login?studio=1` and sign in with your normal credentials. Any non-empty value works. Then fix or deactivate the blueprint page claiming `/login`. ## Why the Escape Hatch Exists [Overwrite Default Pages](https://docs.rual.nl/interfaces/overwrite-default-pages) explains how a [State Page](https://docs.rual.nl/block-types/state%20ui/state_page) whose event URL is `login` replaces the built-in login screen. That override is deliberate and it is complete: when the cluster resolves an incoming request it looks for a blueprint page claiming the path *before* it considers its own built-in pages. A page you define at `/login` therefore shadows the core login screen entirely. Which is exactly what you want, right up to the moment that page stops working. A blank render, a flow error, a half-finished edit, a blueprint someone deactivated: and the one URL every developer needs to sign in is the one URL that no longer works. Without a way around it, fixing the page would require signing in, and signing in would require the page. The escape hatch cuts that loop. Any request carrying a non-empty `studio` query parameter skips user-defined page resolution completely and renders the cluster's own page instead. The guarantee it buys is simple: the studio stays reachable even when an entity has broken its custom page. ## Using It Append `?studio=1` to the URL of the page you need. The built-in version renders, regardless of what any blueprint has claimed for that path. | URL | Renders | | --- | --- | | `https://your-cluster/login?studio=1` | The built-in login screen, even when a custom `/login` page exists. | | `https://your-cluster/passwordreset?studio=1` | The built-in password reset page. | | `https://your-cluster/security?studio=1` | The built-in security page for sessions and multi-factor settings. | If the URL already carries query parameters, add this one alongside them: `/login?redirect=/dashboard&studio=1`. ## The Value Does Not Matter The cluster never inspects what you pass. It checks only that the parameter is present and non-empty, so `?studio=1`, `?studio=request` and `?studio=please` all behave identically. Elsewhere in this documentation you will see `?studio=request`; that is convention, not a required value. Two forms do *not* trigger the escape hatch, because both read as an empty value: `?studio=` with nothing after the equals sign, and `?studio` with no equals sign at all. Page resolution then proceeds normally and your custom page still wins. If in doubt, pass a real value. ## Where It Sits in Page Resolution Every page request is resolved in a fixed order, and the escape hatch owes its reliability to its position in that order: - **Direct GUID access.** A path starting with `no-uris/` renders that specific page action, used for pages that have no event URL of their own. - **The studio escape hatch.** A non-empty `studio` parameter skips resolution: no blueprint page is looked up at all. - **User-defined pages.** Otherwise the path is resolved against the event URLs of your blueprint pages, and a match wins. - **Built-in pages.** Only when nothing above claimed the request do `login`, `passwordreset` and `security` render from the cluster itself. Because step 2 sits above step 3, the escape hatch never depends on your custom page being loadable, valid, or active. It is not a fallback that runs after a failed render: with the parameter present, the custom page is never consulted at all. A page that crashes, a page mid-edit, a page whose blueprint is deactivated, and a page that does not exist are all the same case here. The path is matched after any optional entity prefix is stripped, and on the trailing segment, so `/login`, `/1/login` and `/anything/login` all reach the built-in login screen. ## Recovering From a Broken Login Page - Open `https://your-cluster/login?studio=1`. The built-in login screen renders. - Sign in with your normal credentials. The escape hatch decides which page you get and nothing else; authentication is unchanged. - Open [RUAL Studio](https://docs.rual.nl/interfaces/rual-studio) and find the blueprint holding the State Page whose event URL is `login`. **View Pages** under User Interfaces lists every state page on the cluster, which is the quickest way to find which blueprint claimed the path. - Fix the flow, or hand the path back to the cluster while you work: deactivate the blueprint, or clear the event URL on the State Page. - [Deploy](https://docs.rual.nl/deployment/how-to-deploy). Until you do, production keeps serving the previously deployed version of the page. Because the parameter only ever affects the request that carries it, you can keep working against `?studio=1` for as long as the repair takes, without changing what your users see. ## What It Does Not Do - **It is not an authentication bypass.** The built-in login page is still a login page: valid credentials are required, and the standard protections continue to apply, including blocking repeated failed attempts against one account and mass authentication attempts from a single IP address. - **It does not disable your custom page.** Only the request carrying the parameter is affected. Every other visitor keeps reaching the custom page, working or broken, until you fix or deactivate it. - **It is not a general page bypass.** On paths other than `login`, `passwordreset` and `security` the parameter still skips resolution, but there is no built-in page behind it: signed-out visitors get the built-in login screen, signed-in visitors get a 404. ## Behaviour Across Core Versions The Go core reproduces the Node behaviour deliberately: Node performs the same default fallback whenever the studio query parameter is present. The recovery URL therefore works identically on a Node cluster and a Go cluster, which matters precisely because you reach for it when something else has already gone wrong. ## Where to Go Next - [Overwrite Default Pages](https://docs.rual.nl/interfaces/overwrite-default-pages): how to replace the login, password reset and security pages in the first place. - [Building a Login Page](https://docs.rual.nl/tutorials/login-page): building the custom login page this escape hatch protects you from. - [Building User Authentication](https://docs.rual.nl/tutorials/user-authentication): a full custom login flow with tokens, 2FA and scope checks. - [Getting Access](https://docs.rual.nl/cluster/getting-access): receiving credentials and your first sign-in to a cluster. - [Common Issues](https://docs.rual.nl/troubleshooting/common-issues): other ways a cluster can appear unreachable and what to check first. ## Frequently asked **How do I sign in when my custom RUAL login page is broken?** Append ?studio=1 to the login URL, for example https://your-cluster/login?studio=1. Any non-empty studio query parameter bypasses user-defined page resolution and forces the built-in login page to render, where you sign in with your normal credentials. **Is the RUAL studio parameter an authentication bypass?** No. It only decides which page renders. The built-in login page still requires valid credentials, and the standard protections continue to apply, including blocking repeated failed attempts against one account and mass authentication attempts from a single IP address. **Does the studio parameter affect other visitors of my RUAL cluster?** No. Only the request carrying the parameter is affected. Every other visitor keeps reaching the custom page, working or broken, until you fix or deactivate it. **Which RUAL pages does the studio escape hatch work on?** login, passwordreset and security, because those are the paths with a built-in page behind them. On other paths the parameter still skips resolution, but signed-out visitors get the built-in login screen and signed-in visitors get a 404. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Tutorials · RUAL Documentation" description: "End-to-end builds. Each tutorial starts from an empty blueprint and ends with something running." canonical: https://docs.rual.nl/tutorials language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Tutorials End-to-end builds. Each tutorial starts from an empty blueprint and ends with something running. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Building User Authentication · RUAL Documentation" description: "Login, tokens, 2FA and the scope checks around them." canonical: https://docs.rual.nl/tutorials/user-authentication language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Building User Authentication Sign-in, sessions, scopes, and gated pages for your app. The complete authentication system on a RUAL cluster. Every RUAL cluster already has accounts, sign-in, sessions, and token management built in. Your job is to wire your app's pages and APIs to that system: not to build one. This tutorial assembles the full picture. ## What the Cluster Handles for You - User accounts with passwords (and optional 2FA), created by an administrator. See [User Access Management](https://docs.rual.nl/cluster/user-access-management). - The default **login**, **password reset**, and **security** pages, ready at your cluster URL. See [Overwrite Default Pages](https://docs.rual.nl/interfaces/overwrite-default-pages). - Sessions and `access_token`s (14-day validity, auto-extending), accepted in five locations. See [API Guide, Authentication](https://docs.rual.nl/cluster/api-guide#authentication). ## Deciding Who Can See What Every page and API gets its audience from a **scope**: | Scope | Who passes | Use for | | --- | --- | --- | | `*public` | Anyone, no token | Landing pages, status endpoints, public content | | `*loggedin` | Any valid `access_token` | Member areas, app pages, private APIs | | Custom scope (e.g. `billing_read`) | Users you granted that scope | Role-gated sections (admin, finance, support) | Set scopes per page block or API block via its lock icon: full mechanics in [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control). Grant custom scopes to users on their user detail page (`custom_scopes`) or bundle them in user groups. See [User Roles Explained](https://docs.rual.nl/cluster/user-roles-explained). ## The Sign-In Flow - A visitor hits a page scoped `*loggedin` without a token → the cluster serves the login page. - They sign in (password, plus 2FA code when enabled) → the cluster issues an `access_token` and redirects back. - The token travels as a cookie in the browser, or as a header/query param from your own frontend. - Sign-out invalidates the token immediately; users manage their sessions on the default **security** page. Building your own frontend? Call the sign-in API directly (`POST /api/v1/auth/signin`) and store the token. See the [API workflows](https://docs.rual.nl/cluster/api-guide#workflows). For 2FA-enforced clusters, pass the current code from the user's authenticator. ## Knowing Who's Calling Inside any authenticated flow, the `user_current` block gives you the calling user's document: guid, name, scopes. Typical uses: - Filter data to the caller (`owner_guid = user.guid` in your queries. The denormalized owner pattern from [Storages](https://docs.rual.nl/blueprints/storage)). - Branch on scope: allow admins to a management view, others to their own data. - Audit: write the actor onto every document you create. ## Gating Pages and Sections - Public marketing pages: scope `*public`. - The app shell and member pages: scope `*loggedin`: unauthenticated visitors bounce to login automatically. - Admin-only sections: a custom scope granted to admins only. - Locked yourself out with a custom login page? Recover with `?studio=request`. See [Overwrite Default Pages](https://docs.rual.nl/interfaces/overwrite-default-pages). ## Hardening Checklist - Enable 2FA for everyone who can reach Studio or admin scopes. - Keep `*public` for content that truly is public. Check every page and API block once before launch. - Prefer custom scopes over checking usernames in flows. - Review the failed-login blocking and rate limits in [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control). - Audit access regularly on the User Activity page. See [Auditing User Access](https://docs.rual.nl/cluster/user-access-management#auditing). ## Next Steps - [Getting Access](https://docs.rual.nl/cluster/getting-access): Accounts, first login, cluster URLs, and API tokens. - [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control): Scopes, rate limits, and throttling in depth. - [User Roles Explained](https://docs.rual.nl/cluster/user-roles-explained): Permissions, scopes, and user groups by role. ## Frequently asked **Do I need to build a login system in RUAL?** No. Every cluster already has accounts, sign-in, sessions and token management built in, including default login, password reset and security pages. Your job is to wire your app's pages and APIs to that system through scopes, not to build one. **How do I restrict a RUAL page to logged-in users?** Set the *loggedin scope on the page block through its lock icon. Unauthenticated visitors are then served the login page automatically, and after sign-in the cluster issues an access token and redirects them back. **How do I know which user is calling my RUAL flow?** The user_current block gives you the calling user's document with its guid, name and scopes. Use it to filter data to the caller in your queries, to branch on scope, or to write the actor onto every document you create. **How long does a RUAL access token stay valid?** 14 days, extended automatically while actively used. Sign-out invalidates the token immediately, and users manage their sessions on the default security page. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Creating a REST API · RUAL Documentation" description: "A complete CRUD API with validation, paging and error replies." canonical: https://docs.rual.nl/tutorials/rest-api language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Creating a REST API A complete resource API with list, read, create, update, and delete: proper methods, statuses, validation, and scopes. This tutorial builds a full REST resource: `GET/POST /api/v1/tasks` and `GET/PUT/DELETE /api/v1/tasks/{guid}`. It assumes the quickstart. If you haven't registered an endpoint before, do [Build Your First API](https://docs.rual.nl/cluster/api-quickstart) first. ## The Shape of a Resource | Operation | Method + URI | Success status | | --- | --- | --- | | List | `GET /api/v1/tasks` | 200 + array | | Create | `POST /api/v1/tasks` | 201 + created document | | Read one | `GET /api/v1/tasks/{guid}` | 200 + document, 404 if unknown | | Update | `PUT /api/v1/tasks/{guid}` | 200 + updated document, 404 if unknown | | Delete | `DELETE /api/v1/tasks/{guid}` | 200 or 204, 404 if unknown | Each row is one API block ([`on_startup_register_uri_get`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_get), `_post`, `_put`, `_delete`) feeding one handler function. The same trigger → work → reply shape from [Block Templates](https://docs.rual.nl/block-types/block-templates#api-endpoint). ## GET /tasks, List with Filters - Trigger [`on_startup_register_uri_get`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_get) with URI `tasks` → handler function. - Build the query: optional `status` param from [`httpconnection_get_params`](https://docs.rual.nl/block-types/http%20connection/httpconnection_get_params) → [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) → [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) → `query_and`, with [`query_sort_field`](https://docs.rual.nl/block-types/query/query_sort_field) on `_meta.created` descending. - `function_search` on the `tasks` storage; respect `limit`/`offset` params for pagination. - Reply with [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json): code 200, data = the results array. ## POST /tasks, Create with Validation - Trigger [`on_startup_register_uri_post`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_post) with URI `tasks`. - Validate required fields from the request body with [`condition_not_empty_value`](https://docs.rual.nl/block-types/condition/condition_not_empty_value) per field → `branch`; on the false path reply 400 with `{"error": "title is required"}` (see [validation template](https://docs.rual.nl/block-types/block-templates#data-validation)). - Map the body to document fields with [`mutations_set_bp_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_set_bp_field_multiple), then [`function_create_document_from_mutations`](https://docs.rual.nl/block-types/storage/function_create_document_from_mutations). - Reply 201 with the created document (it now has its `_meta.guid`). ## GET /tasks/{guid}, Read One - The URI block exposes the guid as a param; fetch with [`function_get_document`](https://docs.rual.nl/block-types/storage/function_get_document) on the tasks storage. - On its `found` condition: reply 200 with the document. Otherwise: reply 404 with `{"error": "not found"}`. Wire both paths from the success/error pins (see [error handling](https://docs.rual.nl/blueprints/block-execution#error-handling)). ## PUT and DELETE - **PUT**: same fetch-first guard, then [`function_update_document_mutations`](https://docs.rual.nl/block-types/storage/function_update_document_mutations) with the changed fields. Reply 200 with the updated document; 404 when missing. - **DELETE**: soft-delete with [`function_remove_document`](https://docs.rual.nl/block-types/storage/function_remove_document) (recoverable), or hard-delete with [`function_delete_document`](https://docs.rual.nl/block-types/storage/function_delete_document) when you mean it. See [document lifecycle](https://docs.rual.nl/blueprints/storage#document-lifecycle). Reply 200 with a small confirmation object. ## Scopes and Testing - Scope the collection endpoints with the lock icon: `*public` for open reads, `*loggedin` or a custom scope for writes. See [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control). - Activate the blueprint, then test every row of the table with curl: commands in [API workflows](https://docs.rual.nl/cluster/api-guide#workflows). - When the endpoint 404s right after activation, it's almost always a scope or an inactive blueprint. See [Common Issues](https://docs.rual.nl/troubleshooting/common-issues#api-not-activating). ## Next Steps - [API Guide](https://docs.rual.nl/cluster/api-guide): Conventions, error codes, and rate limits for your API. - [Storages](https://docs.rual.nl/blueprints/storage): The queries and mutations behind every row of the table. - [Block Templates](https://docs.rual.nl/block-types/block-templates): The endpoint and validation shapes in one view. ## Frequently asked **How do I build a CRUD REST API in RUAL?** One API block per operation: GET and POST /api/v1/tasks for list and create, and GET, PUT and DELETE /api/v1/tasks/{guid} for read, update and delete. Each on_startup_register_uri block feeds one handler function that does the work and replies with the right status: 200 for reads, 201 for creates, 404 for unknown guids. **How do I validate the request body in a RUAL API?** Check each required field with its own condition_not_empty_value block into a branch, and reply 400 on the false path before anything is stored. On the success path, map the body to document fields with mutations_set_bp_field_multiple and create the document from those mutations. **Should a RUAL DELETE endpoint remove or delete the document?** Your choice: function_remove_document soft-deletes and stays recoverable, while function_delete_document hard-deletes when you mean it. Reply 200 with a small confirmation object either way. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Building a Dashboard with Charts · RUAL Documentation" description: "Aggregate storage data and render it as a live dashboard." canonical: https://docs.rual.nl/tutorials/dashboard-charts language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Building a Dashboard with Charts Aggregate your storage data, render it live on a page, and keep it fast. The dashboard pattern end to end. A dashboard is a page backed by a few well-chosen queries: counts, sums, recent items, and grouped stats. This tutorial assembles one: data first, then rendering, then keeping it fast. ## Step 1, Get the Data Dashboards read. Never write. Build a function per metric (or one function returning an object with all of them): - **Counts**: `function_search` with the `hits` out-pin (total matching documents), one per filtered query: open tasks, shipped orders, this week's signups. - **Recent items**: `function_search` + [`query_sort_field`](https://docs.rual.nl/block-types/query/query_sort_field) on `_meta.created` descending + `limit` 10. See [search tips](https://docs.rual.nl/blueprints/storage#search-tips). - **Grouped stats**: aggregations where the storage supports them; for anything heavy, precompute counters on write instead (see performance below). Return everything from one function as an object built with [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields). One call, one payload for the page. ## Step 2, Render the Page - A `state_page` for the dashboard URL, scoped for your team (`*loggedin` or a custom scope). - On load, run the metrics function from the render flow and pass results into state elements: number tiles (styled elements), a recent-items list (iterate the array), and your chart component. - For real charts, mount a custom React component in the page and feed it the metrics object. See [Components](https://docs.rual.nl/interfaces/components) and [RUAL Library](https://docs.rual.nl/interfaces/rual-library) for passing data to components. - Want it live? Real-time search updates the metrics as documents change: worth it on shared boards, overkill on quiet data (see [real-time vs polling](https://docs.rual.nl/architecture/choosing-approaches#realtime-vs-polling)). ## Step 3, Keep It Fast - **Cache the metrics function** with a `cache_key` on `function_search` and a short TTL via Redis. The dashboard loads instantly even with thousands of documents. See [caching patterns](https://docs.rual.nl/blueprints/common-patterns#caching-strategies). - **Precompute counters on write**: increment a stats document on each create ([`mutations_increment_by_field`](https://docs.rual.nl/block-types/mutations/mutations_increment_by_field)) instead of counting at read time; reads become a single [`function_get_document`](https://docs.rual.nl/block-types/storage/function_get_document). - **Avoid dashboard-side loops**. If the page needs per-item computation, do it in the function with `array_map`, not in the render. - Watch out for the aggregation traps in [Common Pitfalls](https://docs.rual.nl/blueprints/common-pitfalls). ## Checklist - One metrics function returning one object. - Sort + limit on every "recent" list. - Cache key on expensive searches; counters for hot stats. - Scope set intentionally (no accidental `*public` dashboards). ## Next Steps - [Components](https://docs.rual.nl/interfaces/components): Mounting custom React components in RUAL pages. - [Common Blueprint Patterns](https://docs.rual.nl/blueprints/common-patterns): Caching and transformation recipes for the metrics function. - [Storage Examples](https://docs.rual.nl/blueprints/storage-examples): Data models that aggregate well. ## Frequently asked **How do I build a dashboard page in RUAL?** Build one function that gathers every metric, counts via the hits out-pin of function_search, recent items via a sort on _meta.created with a limit, grouped stats via aggregations, and return everything as one object built with object_new_fields. The page runs that function on load and passes the results into state elements or a chart component. **How do I show charts on a RUAL page?** Mount a custom React component in the page and feed it the metrics object. Number tiles and recent-items lists can be plain state elements; real charts go through the component. **How do I keep a RUAL dashboard fast?** Cache the metrics function with a cache_key on function_search and a short TTL in Redis, precompute counters on write with mutations_increment_by_field instead of counting at read time, and do per-item computation in the function with array_map rather than in the render. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "File Upload & Processing · RUAL Documentation" description: "Accept an upload, store it as an asset and process it safely." canonical: https://docs.rual.nl/tutorials/file-upload-processing language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # File Upload & Processing Accept files through forms and APIs, store them as assets, and process them safely in the background. Uploads come in two flavors: a user dropping a file into your page, and a system pushing a file to your API. Both end up as cluster assets, and heavy processing belongs in the queue. This tutorial covers the whole pipeline. ## Upload via a Page (User-Facing) - Build a form with `state_form` and a file-capable input from the state elements. See the form pattern in [Block Templates](https://docs.rual.nl/block-types/block-templates#form-handling). - On submit. Read the form with [`state_form_get`](https://docs.rual.nl/block-types/state%20ui/state_form_get); the file arrives with its name, type, and size. - Store it via the asset upload API (`POST /api/v1/_system/assets/stream`) or a file-handling block, into `public` (world-readable) or `system_assets` (private): choose per [public vs private](https://docs.rual.nl/blueprints/assets#public-vs-private). - Record the asset guid/path on a document so you can find it later (e.g. `photo` on a profile document). ## Upload via an API (Machine-Facing) - Register [`on_startup_register_uri_post`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_post) with your upload URI. See [the upload template](https://docs.rual.nl/block-types/block-templates#file-uploads). - Validate: content type, size limit, required metadata. Reject with a 400 before storing anything. - Store as an asset; reply 200 with `{"guid": "...", "path": "..."}` so the caller can reference it. - Scope it deliberately: uploads are never `*public` by accident; use `*loggedin` or a custom scope with a secret in the path. ## Processing: Do It in the Background Resizing, parsing, importing, converting. Never inline in the request. The pattern: - The upload flow stores the file and enqueues a processing job with [`function_custom_execute_from_queue`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute_from_queue). See [Queue](https://docs.rual.nl/blueprints/queue). - A queue worker flow picks it up: reads the file with [`file_from_assets`](https://docs.rual.nl/block-types/files/file_from_assets), transforms it (e.g. resize/convert with the image blocks), writes results back to a document or a new asset. - Mark progress on the job document (`status: queued → processing → done|failed`) so users can poll or a page can show state. - Log failures to the console and storage. The error-pins pattern from [Block Execution](https://docs.rual.nl/blueprints/block-execution#error-handling). ## Safety Rules - Never trust the client's filename for the stored path, generate your own. - Cap file size at the API (the asset API rejects oversize with `FILE_TOO_BIG`) and validate type by content, not extension. - Keep user uploads out of `public` unless they are truly public: private folders + scoped serving, per [Serving Protected Files](https://docs.rual.nl/blueprints/assets#serving-protected-files). - Import pipelines should be idempotent: re-running the same file updates the same output document (upsert by source guid). ## Next Steps - [Assets](https://docs.rual.nl/blueprints/assets): Asset paths, APIs, and access rules in depth. - [Queue](https://docs.rual.nl/blueprints/queue): Background processing for the heavy lifting. - [Block Templates](https://docs.rual.nl/block-types/block-templates): The upload template this pipeline extends. ## Frequently asked **How do I accept file uploads in RUAL?** Two flavors: a page form with state_form and a file-capable input, read on submit with state_form_get, or a machine-facing API endpoint registered with on_startup_register_uri_post. Both store the file as a cluster asset, and you record the asset guid or path on a document so you can find it later. **How do I process uploaded files in RUAL without blocking the request?** The upload flow stores the file and enqueues a processing job with function_custom_execute_from_queue. A queue worker flow reads the file with file_from_assets, transforms it, and writes results back to a document or a new asset, marking progress on a job document so users can poll the state. **What safety rules apply to RUAL file uploads?** Never trust the client's filename for the stored path, validate the type by content and not by extension, keep user uploads out of public storage unless they are truly public, and make import pipelines idempotent by upserting on the source guid. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Email System Setup · RUAL Documentation" description: "Templates, sending and delivery handling for transactional email." canonical: https://docs.rual.nl/tutorials/email-system language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Email System Setup From zero to sending: tokens, from-addresses, templates, flows that trigger mail, and deliverability discipline. Email on RUAL runs through Postmark: the cluster holds your server token, blueprints trigger sends, and you keep templates and recipients in storage. This tutorial sets the whole thing up in order. ## Step 1, Prerequisites - Create a Postmark account and a **server token**. - Add it as a **secure** system setting named `postmark_token`. See [Email configuration](https://docs.rual.nl/blueprints/system-settings-reference#email-setup). - Set `email_default_from` to an address on your verified sender domain (e.g. `noreply@yourdomain.com`). - Optionally set `messagebird_originator` if you also send SMS for 2FA. From this point, platform emails (password resets) already work. The rest is your own mail. ## Step 2: Sending from a Flow - Pick the trigger: form submit, `storage_event` (document created/updated), [`schedule_repeating_event`](https://docs.rual.nl/block-types/events/schedule_repeating_event) (digests), or another API call. - Collect what the mail needs: recipient profile, order, or template values: usually one `function_search` or [`function_get_document`](https://docs.rual.nl/block-types/storage/function_get_document). - Use the Postmark send blocks (search `postmark` in the block library) with to/subject/body from your data. The shape from [the email template](https://docs.rual.nl/block-types/block-templates#email-sending). - Handle the send result: success/error pins → [`function_console_log`](https://docs.rual.nl/block-types/logging/function_console_log) while developing, and onto a mail-log document in production. ## Step 3: Templates That Survive Edits - Keep subjects and bodies in storage (a `mail_templates` storage) with keys like `welcome`, `reset-reminder`, `weekly-digest`: edit content without touching the blueprint. - Build the final body with [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields) + text blocks, merging template with per-mail data (name, links, amounts). - Version links with full URLs (`https://your-cluster/...`): emails have no relative paths. ## Step 4: Bulk and Scheduled Mail - Never loop-and-send inline in a request: enqueue per-recipient jobs with [`function_custom_execute_from_queue`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute_from_queue) and let the queue drain them. See [Queue](https://docs.rual.nl/blueprints/queue). - Scheduled digests: [`schedule_repeating_event`](https://docs.rual.nl/block-types/events/schedule_repeating_event) → gather since-last-run (range query on `_meta.created`) → one mail per user. - Idempotency: record `last_sent_at` per user/digest so retries never double-send. ## Deliverability Discipline - Verify the sender domain in Postmark before sending anything (SPF/DKIM handled there). - One clear from-address per purpose; don't rotate from-addresses mid-thread. - Log every send (to, template, status, error) to a storage. You will need it the first time a user says "I never got it". - Test with a private alias first; Postmark's activity view confirms delivery before you blame the blueprint. ## Next Steps - [System Settings Reference](https://docs.rual.nl/blueprints/system-settings-reference): postmark_token, email_default_from, and secure values. - [Queue](https://docs.rual.nl/blueprints/queue): Bulk and scheduled sending without blocking requests. - [Storage Events](https://docs.rual.nl/blueprints/storage-events): Triggering mail on document changes. ## Frequently asked **How do I set up email sending on a RUAL cluster?** Create a Postmark server token, add it as a secure system setting named postmark_token, and set email_default_from to an address on your verified sender domain. From that point platform emails like password resets already work, and blueprints send their own mail with the Postmark send blocks. **How do I send bulk or scheduled email in RUAL?** Never loop-and-send inline in a request: enqueue per-recipient jobs with function_custom_execute_from_queue and let the queue drain them. Scheduled digests run from schedule_repeating_event, gathering what changed since the last run, and recording last_sent_at per user so retries never double-send. **How do I edit email content without changing blueprints in RUAL?** Keep subjects and bodies in a mail_templates storage with keys like welcome or weekly-digest, and build the final body by merging the template with per-mail data. Use full URLs for links, because emails have no relative paths. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Third-Party Integrations · RUAL Documentation" description: "Call external APIs, keep credentials out of the canvas and handle failures." canonical: https://docs.rual.nl/tutorials/third-party-integrations language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Third-Party Integrations Calling external APIs from blueprints, receiving webhooks, storing credentials safely, and keeping integrations alive. Most integrations are one of two shapes: your blueprint calls their API, or their service calls yours (a webhook). RUAL handles both with the same primitives. This tutorial sets up a real one end to end and covers the operational rules that keep it healthy. ## Outbound: Calling Their API - **Credentials first**. When the service has a provider type, configure a [service provider](https://docs.rual.nl/blueprints/service-providers) and name it on the block: the key stays on the cluster and never enters the run. For anything else, store the key as a **secure** system setting (e.g. `stripe_secret_key`) and read it with `get system setting`. Never in the blueprint. See [custom keys](https://docs.rual.nl/blueprints/system-settings-reference#custom-keys). - **Build the request** with the http request blocks: URL, method, headers (auth from the setting), body from [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields) or JSON. - **Parse the response**: map only the fields you need into your documents with [`object_update_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_update_fields); don't mirror their whole payload. - **Handle failure explicitly**: success/error pins → log, retry via queue for transient errors (429/5xx), alert path for permanent ones. See [error handling](https://docs.rual.nl/blueprints/block-execution#error-handling). Many popular services have ready-made block groups (postmark, mailchimp, discord, twilio). Search the service name in the block library before building raw http requests. ## Inbound: Receiving Webhooks - Register an endpoint with [`on_startup_register_uri_post`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_post): your public webhook URL is `https:// /api/v1/ `. - **Verify the sender**: shared-secret signature header, HMAC check, or an unguessable URI segment. Treat every unverified payload as hostile. - **Reply fast** (200 with a small ack), then do the real work in the [queue](https://docs.rual.nl/blueprints/queue): senders time out and retry otherwise, giving you duplicate events. - **Deduplicate** by event id: store received ids, skip ones you've seen (webhooks retry). ## Keeping Data in Sync - **Pull**: [`schedule_repeating_event`](https://docs.rual.nl/block-types/events/schedule_repeating_event) polls their list endpoint on an interval and upserts by external id (update if exists, create if new. See [Storages](https://docs.rual.nl/blueprints/storage)). - **Push**: react to local changes with [storage events](https://docs.rual.nl/blueprints/storage-events) and forward them to the service. - **Conflict rule**: pick one system of record per field and write it down in the flow's comments. Two-way sync without it always corrupts something. ## Operational Rules - Rotate credentials by editing the [service provider](https://docs.rual.nl/blueprints/service-providers) or the system setting. Every flow picks it up next run, no blueprint edits. - Log every outbound call (endpoint, status, duration) during rollout; sample it later. - Respect their rate limits with queue debounce instead of bursts. - Build a tiny health check: a scheduled call that verifies the token still works, and alerts via your own email flow ([Email System Setup](https://docs.rual.nl/tutorials/email-system)). ## Next Steps - [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control): Securing the webhooks you expose. - [Queue](https://docs.rual.nl/blueprints/queue): Retries, debounce, and background sync work. - [Common Blueprint Patterns](https://docs.rual.nl/blueprints/common-patterns): Auth and error-handling recipes for your calls. ## Frequently asked **How do I call an external API from a RUAL blueprint?** Store the API key as a secure system setting and read it with get system setting, build the request with the http request blocks, and map only the fields you need into your documents. Search the block library for the service name first: postmark, mailchimp, discord and twilio have ready-made block groups. **How do I receive webhooks in RUAL?** Register an endpoint with on_startup_register_uri_post, verify the sender with a signature header, HMAC check or unguessable URI segment, reply fast with a small 200 ack, and do the real work in the queue. Deduplicate by event id, because senders time out and retry. **How do I keep RUAL data in sync with an external service?** Pull with a schedule_repeating_event that polls their list endpoint and upserts by external id, and push by reacting to local changes with storage events. Pick one system of record per field and write it down, because two-way sync without that rule always corrupts something. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Building AI Endpoints · RUAL Documentation" description: "Add Claude to a blueprint: messages, structured JSON replies, streaming, tools, cost control and batches." canonical: https://docs.rual.nl/tutorials/ai-endpoints language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Building AI Endpoints Add Claude to a blueprint: create an Anthropic connection, send messages from an API flow, force structured JSON replies, stream to the browser, control cost, and run bulk work as batches. This tutorial builds an AI-backed endpoint: a POST API that takes a text, asks Claude, and replies with clean JSON. It assumes you can already register an endpoint and reply in JSON; if not, do [Build Your First API](https://docs.rual.nl/cluster/api-quickstart) first. You need an Anthropic API key. ## 1. The Connection Every Claude block takes a connection pin. Create it once per flow with [`function_create_anthropicconnection`](https://docs.rual.nl/block-types/anthropic/function_create_anthropicconnection) (open connection): - `api_key`: your Anthropic key. Keep it out of the canvas by storing it as a system setting and reading it with [`value_get_system_key_select`](https://docs.rual.nl/block-types/system%20settings/value_get_system_key_select) (get setting). - `default_model`: the model every message uses unless the block overrides it. - `daily_cap_usd` and `concurrency_cap`: hard guardrails on spend and parallel calls, enforced by the connection itself. The block has a `success` condition pin and an `error` value pin. Branch on them: a message sent without a working connection fails further down the flow. ## 2. Request to AI to JSON Reply The dominant API shape in RUAL blueprints works for AI too: a trigger runs the chain, the connection and prompt feed the message block, and a JSON reply answers the caller. The prompt here is the `text` field of the request body, read with [`httpconnection_get_body`](https://docs.rual.nl/block-types/http%20connection/httpconnection_get_body) (get body) and [`object_field_getter_multiple`](https://docs.rual.nl/block-types/object%2Cfields/object_field_getter_multiple) (get fields). The standard AI endpoint shape: a [function trigger](https://docs.rual.nl/block-types/globals/trigger_custom_function) runs the chain, [open connection](https://docs.rual.nl/block-types/anthropic/function_create_anthropicconnection) creates the Anthropic connection, and [send message](https://docs.rual.nl/block-types/anthropic/anthropicconnection_message) asks Claude with the request body text as the prompt, read off the body by [get fields](https://docs.rual.nl/block-types/object/object_field_getter_multiple). An [output template](https://docs.rual.nl/block-types/anthropic/anthropicconnection_build_schema) on the output_schema pin keeps the answer valid JSON, and [reply in JSON](https://docs.rual.nl/block-types/http%2520connection/httpconnection_set_json) returns it to the caller with a 200. [![Studio canvas example for the send message block: from API request to AI answer to JSON reply.](https://docs.rual.nl/canvas-examples/anthropicconnection_message.png)](https://docs.rual.nl/canvas-examples/anthropicconnection_message.png?v=3) On the canvas: [`anthropicconnection_message`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_message) (send message) takes the connection, the prompt and an optional `system` text, and outputs the reply as `text` plus token `usage` and a `stop_reason`. [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) (reply in JSON) sends the answer back with status 200. Always branch on the message block's `success` pin and reply with an error body on the false path, following the pattern in [Error Handling](https://docs.rual.nl/blueprints/common-patterns#error-handling). ## 3. Structured JSON Replies Free text is hard to process further. Two blocks turn the reply into dependable data: - [`anthropicconnection_build_schema`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_build_schema) (build output template): pick the fields Claude should return and describe each one, no JSON Schema knowledge needed. Wire its output to the message block's `output_schema` pin and the reply text is valid JSON in that shape. - [`anthropicconnection_parse_json`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_parse_json) (parse JSON): recovers and parses JSON from messy text (code fences, prose around the object, truncation) without an API call. It reports `repaired` when it had to fix the input, so you can log how often that happens. With an output template wired, the text pin already parses cleanly; keep the parse block as the safety net before you feed the reply into object blocks or storage. ## 4. Streaming to the Browser [`anthropicconnection_message_stream`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_message_stream) (stream message) sends the same prompt but pushes the reply token by token to a WebSocket `channel` while it generates, then returns the full text when done. Use it for chat-like pages where waiting for the whole answer feels broken; use the plain message block everywhere else, it is simpler to debug. ## 5. Classify and Route [`anthropicconnection_classify`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_classify) (classify) sorts an input into exactly one of your labels, with a `matched` condition for "none of them fit". Feed the `label` into a [`value_switch`](https://docs.rual.nl/block-types/value/value_switch) (switch) to route support tickets, form submissions or webhook payloads down the right branch without writing conditions by hand. ## 6. Tools, Claude Calling Your Functions Tools let Claude call back into your blueprint while it composes an answer. Build one with [`anthropicconnection_build_tool`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_build_tool) (build tool): name, description, parameters as a schema, and the function that runs when Claude calls it. Combine tools with [`anthropicconnection_tool_multiple`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_tool_multiple) (multiple), pick one conditionally with [`anthropicconnection_tool_branch`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_tool_branch) (tool if) or [`anthropicconnection_tool_switch`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_tool_switch) (tool switch), and wire the result to the message block's `tools` pin. Ready-made tools cover web search ([`anthropicconnection_tool_web_search`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_tool_web_search)) and code execution ([`anthropicconnection_tool_code_execution`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_tool_code_execution)). The `max_iterations` pin caps how many tool round trips one message may make. ## 7. Cost Control - [`anthropicconnection_count_tokens`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_count_tokens) (count tokens) prices a prompt before you send it: input tokens, estimated total cost, free to call. Gate expensive calls on it. - [`anthropicconnection_preflight`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_preflight) (preflight document) converts a PDF, image or spreadsheet to clean Markdown with a cheap model, so the expensive model reads far fewer tokens. [`anthropicconnection_preflight_batch`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_preflight_batch) does the same for a list of files. - The message block's `cache` pin reuses earlier context instead of paying for it again, and [`anthropicconnection_get_usage`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_get_usage) (get usage) reports total spend per connection. - The connection's daily cap is the last line of defense: set it on every production blueprint. ## 8. Bulk Work as Batches [`anthropicconnection_batch_create`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_batch_create) (create batch) submits many prompts as one asynchronous batch at half the standard price, with results within 24 hours. Wire its `on_finished` function pin to be called when the batch completes (the system polls in the background), or poll yourself with [`anthropicconnection_batch_get`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_batch_get) (get batch) from a scheduled flow. Batches are the right shape for nightly enrichment, classification backfills and report generation; anything the user waits for belongs in a plain message call. ## Notes From Practice - Long AI calls belong behind the queue: hand the work to [`function_custom_execute_from_queue`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute_from_queue) (execute in queue) and reply to the caller immediately. See [Queue](https://docs.rual.nl/blueprints/queue). - Expose AI endpoints with a scope and a rate limit; an unprotected `/api/ask` is an open wallet. See [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control). - Conversations that remember earlier turns use the memory blocks ([`anthropicconnection_create_memory`](https://docs.rual.nl/block-types/anthropic/anthropicconnection_create_memory) and friends) on the message block's `memory` pin. - [Common Blueprint Patterns](https://docs.rual.nl/blueprints/common-patterns): The endpoint, validation and error-handling shapes this tutorial builds on. - [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control): Scopes and rate limits for your AI endpoint. - [Block Templates](https://docs.rual.nl/block-types/block-templates): Copyable block combinations, including the API endpoint shape. ## Frequently asked **How do I call Claude from a RUAL blueprint?** Create a connection with function_create_anthropicconnection using your Anthropic API key, then wire that connection to anthropicconnection_message with your prompt. The message block returns the reply text, token usage and a success condition to branch on. **How do I get structured JSON from Claude in RUAL?** Build an output template with anthropicconnection_build_schema, describing each field Claude should return, and wire it to the message block's output_schema pin. The reply text is then valid JSON in that shape, and anthropicconnection_parse_json repairs any malformed output without another API call. **How do I control what Claude costs in a RUAL blueprint?** Set a daily_cap_usd on the Anthropic connection as a hard limit, price a prompt before sending it with anthropicconnection_count_tokens, and convert documents to Markdown with the cheaper preflight block before sending them to an expensive model. Bulk work goes through anthropicconnection_batch_create at half the standard price. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Building a Login Page · RUAL Documentation" description: "Passwordless login: email a code, verify it into a session token, and the form page around it." canonical: https://docs.rual.nl/tutorials/login-page language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Building a Login Page Passwordless login in two small API flows: email a login code, verify it into a session token, and the form page that ties them together. This tutorial builds passwordless login, the shape real RUAL applications use most: the visitor enters an email address, receives a one-time code, and trades it for a session. No passwords to store, no reset flow to maintain. It assumes you can register an endpoint and reply in JSON; if not, do [Build Your First API](https://docs.rual.nl/cluster/api-quickstart) first. ## 1. Request the Code The first endpoint takes an email address and sends the code. Register a POST URI with [`on_startup_register_uri_post`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_post) (on api post), read the body with [`httpconnection_get_body`](https://docs.rual.nl/block-types/http%20connection/httpconnection_get_body) (get body), and lift the email off it with [`object_field_getter_multiple`](https://docs.rual.nl/block-types/object%2Cfields/object_field_getter_multiple) (get fields). Normalize the address before you use it: [`value_trim`](https://docs.rual.nl/block-types/value/value_trim) and [`value_to_lowercase`](https://docs.rual.nl/block-types/value/value_to_lowercase) keep `Ada@Example.com` and `ada@example.com` from becoming two accounts. [`function_user_request_passwordless_code`](https://docs.rual.nl/block-types/users/function_user_request_passwordless_code) (request passwordless login code) takes the address as `identifier` with `identifier_type` set to `email`, generates the code and mails it. The code itself is never exposed to the flow: there is nothing to log or leak. A [function trigger](https://docs.rual.nl/block-types/globals/trigger_custom_function) runs the endpoint: the request body is read with [get body](https://docs.rual.nl/block-types/http%2520connection/httpconnection_get_body), [get fields](https://docs.rual.nl/block-types/object/object_field_getter_multiple) takes the email off it, and [request passwordless login code](https://docs.rual.nl/block-types/users/function_user_request_passwordless_code) mails the code. A [branch](https://docs.rual.nl/block-types/flow/branch) on the success pin decides between a 200 reply and a 400, both sent with [reply in JSON](https://docs.rual.nl/block-types/json/httpconnection_set_json). [![Studio canvas example for the request passwordless login code block: the request-code half of passwordless login.](https://docs.rual.nl/canvas-examples/function_user_request_passwordless_code.png)](https://docs.rual.nl/canvas-examples/function_user_request_passwordless_code.png?v=3) Branch on the block's `success` pin and answer both paths with [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) (reply in JSON), as on the canvas above. Keep the failure reply vague on purpose: a precise error like "no account with this email" tells a caller exactly which addresses have accounts. ## 2. Verify the Code Into a Session The second endpoint takes the email address and the code, and signs the user in with [`function_user_signin`](https://docs.rual.nl/block-types/users/function_user_signin) (sign in): the address goes to `user_id`, the submitted code to `password`, and an `expiry_date` built with [`date_currentdate`](https://docs.rual.nl/block-types/date%2Ctimezone/date_currentdate) plus [`date_add_days`](https://docs.rual.nl/block-types/date%2Ctimezone/date_add_days) sets how long the session lives. On success the block outputs the `accesstoken` and the `user` object; reply with both, so the caller stores the token and sends it on every later request (see [Adding tokens to your requests](https://docs.rual.nl/cluster/api-guide#token-providing)). Two details of the sign-in block are worth knowing up front: - Its `error` pin distinguishes a wrong code from other failures; log it with [`function_console_log`](https://docs.rual.nl/block-types/logging/function_console_log) (log), but reply with a generic 401. - Accounts with two-factor authentication enabled get `need_2fa` true plus the available `2fa_methods`, so the same endpoint can hand the login off to your second-factor step. ## 3. The Page Side The login page itself is a state page: [`state_form`](https://docs.rual.nl/block-types/state%20ui/state_form) (form) with two [`state_input`](https://docs.rual.nl/block-types/state%20ui/state_input) (input) fields collects the email address and later the code, and the submit action calls your functions. Read what was submitted with [`state_form_get`](https://docs.rual.nl/block-types/state%20ui/state_form_get) (get form data), validate before calling the endpoint (the [form validation pattern](https://docs.rual.nl/blueprints/common-patterns#form-validation)), and store the token from the verify reply in the page state. A full account system with registration, roles and gated pages is walked through in [Building User Authentication](https://docs.rual.nl/tutorials/user-authentication). ## Notes From Practice - Scope both endpoints `*public` (a login page is the one thing nobody is logged in for) and put a rate limit on them; the request-code endpoint sends email, so it is a spam vector. See [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control). - The request block deliberately bypasses the built-in rate limit so a code can be sent right after account creation; your own throttle is the only brake. - Prefer a classic password flow? The same two-endpoint shape fits [`function_user_request_password_reset_token`](https://docs.rual.nl/block-types/users/function_user_request_password_reset_token) and [`function_user_reset_password_with_token`](https://docs.rual.nl/block-types/users/function_user_reset_password_with_token); search those type names in the editor to add them. - Every reply path matters here more than anywhere: an unanswered login attempt looks like a hung page, not an error. - [Building User Authentication](https://docs.rual.nl/tutorials/user-authentication): Registration, roles, scopes and gated pages around this login flow. - [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control): Scopes and rate limits for the two endpoints. - [Block Templates](https://docs.rual.nl/block-types/block-templates): The endpoint and validation shapes in one view. ## Frequently asked **How do I build a passwordless login in RUAL?** Two endpoints: one takes the email address and calls function_user_request_passwordless_code, which mails a one-time code without ever exposing it to the flow. The second takes the address plus the code and calls function_user_signin, which returns an access token and the user object on success. **How do I set how long a RUAL login session lasts?** The sign-in block takes an expiry_date pin. Build it with date_currentdate plus date_add_days, for example seven days, and the issued access token stays valid until that date. **Does RUAL login support two-factor authentication?** Yes. When an account has two-factor enabled, function_user_signin returns need_2fa true plus the available 2fa_methods, so the login endpoint can hand off to your second-factor step before issuing the token. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Building a Reporting API · RUAL Documentation" description: "Aggregations over storage, answered as a CSV or XLSX download or a scheduled email." canonical: https://docs.rual.nl/tutorials/reporting-api language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Building a Reporting API Reports from storage data: aggregate with date histograms and terms, answer with a CSV or XLSX download, and deliver scheduled reports by email through the queue. This tutorial builds a report endpoint: call it with a date range and get back a file, not a page of JSON. It covers the three shapes reports take in practice (a download now, an email later, a scheduled run), all fed by the same aggregation query. It assumes [Storages](https://docs.rual.nl/blueprints/storage) and a registered endpoint; see [Build Your First API](https://docs.rual.nl/cluster/api-quickstart) for the latter. ## 1. The Aggregation Reports group documents instead of listing them. The grouping block picks the dimension: - [`aggregation_type_date_histogram_bp_field`](https://docs.rual.nl/block-types/aggregation/aggregation_type_date_histogram_bp_field) (date histogram) buckets a date field per `interval` (`day`, `week`, `month`): signups per day, revenue per week. - [`aggregation_type_terms_bp_field`](https://docs.rual.nl/block-types/aggregation/aggregation_type_terms_bp_field) (terms) counts documents per distinct value of a field: orders per status, tickets per assignee. - [`aggregation_type_cardinality_bp_field`](https://docs.rual.nl/block-types/aggregation/aggregation_type_cardinality_bp_field) (cardinality) counts distinct values, and [`aggregation_type_value_count_bp_field`](https://docs.rual.nl/block-types/aggregation/aggregation_type_value_count_bp_field) (value count) counts entries. [`query_aggregation`](https://docs.rual.nl/block-types/query/query_aggregation) (aggregation) wraps the grouping block under a `name` and attaches it to the search. Feed it into the `query` pin of [`function_search`](https://docs.rual.nl/block-types/storage/function_search) together with a filter query that limits the range (see the query blocks in [Storages](https://docs.rual.nl/blueprints/storage#building-search-queries)). Wire several aggregations and the studio adds one extra query pin per aggregation on the search block, so one pass over the data can fill a whole dashboard. ## 2. The Download For a file the caller saves directly, build it and reply with it in one flow: A [function trigger](https://docs.rual.nl/block-types/globals/trigger_custom_function) runs the report: a [date histogram](https://docs.rual.nl/block-types/aggregation/aggregation_type_date_histogram_bp_field) groups documents per day, [aggregation](https://docs.rual.nl/block-types/query/query_aggregation) names it and feeds [search](https://docs.rual.nl/block-types/storage/function_search), and [build CSV](https://docs.rual.nl/block-types/files/function_build_csv) turns the result rows into a file that [reply with file](https://docs.rual.nl/block-types/http%2520connection/httpconnection_set_file) downloads to the caller. [![Studio canvas example for the build CSV block: an aggregation straight into a CSV download.](https://docs.rual.nl/canvas-examples/function_build_csv.png)](https://docs.rual.nl/canvas-examples/function_build_csv.png?v=3) On the canvas: [`function_build_csv`](https://docs.rual.nl/block-types/files/function_build_csv) (build CSV) takes the search result rows as `values` plus a `filename`, and [`httpconnection_set_file`](https://docs.rual.nl/block-types/files%2Chttp%20connection/httpconnection_set_file) (reply with file) streams the file to the caller through the [`httpconnection_current_request`](https://docs.rual.nl/block-types/http%20connection/httpconnection_current_request) connection. For spreadsheets with real types and multiple sheets use [`file_to_xlsx`](https://docs.rual.nl/block-types/files/file_to_xlsx) (build XLS/X) instead; the wiring is identical. ## 3. Email and Scheduled Reports A heavy report does not belong inside the request that asked for it. The pattern that scales: - The endpoint validates the request and hands the work to [`function_custom_execute_from_queue`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute_from_queue) (execute in queue), then replies immediately with 202. See [Queue](https://docs.rual.nl/blueprints/queue). - The queued function runs the same aggregation, builds the file, and mails it as an attachment with [`postmarkconnection_send_email_template`](https://docs.rual.nl/block-types/postmark/postmarkconnection_send_email_template) (send email with template), which takes the CSV or XLSX straight on its `file` pin. See [Email System Setup](https://docs.rual.nl/tutorials/email-system) for the connection. - A report that runs on its own schedule (the Monday morning export) starts from [`schedule_repeating_event`](https://docs.rual.nl/block-types/events/schedule_repeating_event) (repeating event) instead of an endpoint. See [Repeating Events](https://docs.rual.nl/blueprints/repeating-events). ## Notes From Practice - Reports over large storages are read-heavy by definition; give the search a `cache_key` so a dashboard refreshed by ten people runs the aggregation once. See [Caching Strategies](https://docs.rual.nl/blueprints/common-patterns#caching-strategies). - Branch on the search block's `success` pin and reply with an error body on failure, per [Error Handling](https://docs.rual.nl/blueprints/common-patterns#error-handling). A report that silently returns an empty file is worse than none. - Scope report endpoints tightly: they aggregate exactly the data most callers should not bulk-download. `*loggedin` at minimum, a custom scope for anything financial. See [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control). - Date histograms group on the stored date field; `_meta.created` is the one every document already has. - [Storages](https://docs.rual.nl/blueprints/storage): The query blocks that limit what the aggregation reads. - [Queue](https://docs.rual.nl/blueprints/queue): Hand slow reports to the queue and reply immediately. - [Email System Setup](https://docs.rual.nl/tutorials/email-system): The Postmark connection that mails the finished report. ## Frequently asked **How do I build a CSV report from storage data in RUAL?** Group the documents with an aggregation block such as aggregation_type_date_histogram_bp_field, attach it to a search with query_aggregation, feed the result rows to function_build_csv, and reply with the file through httpconnection_set_file. **How do I email a report on a schedule in RUAL?** Start the flow from schedule_repeating_event instead of an endpoint, run the same aggregation, and mail the built file with postmarkconnection_send_email_template, which accepts the CSV or XLSX directly on its file pin. Reports triggered by a request go through the queue so the API can reply immediately. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Handling City Filter Uploads · RUAL Documentation" description: "Parse uploaded CSV or XLSX locations and keep only the rows inside your city or radius." canonical: https://docs.rual.nl/tutorials/city-filter-uploads language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Handling City Filter Uploads Accept a CSV or XLSX of locations over an API, parse it into rows, keep only what falls inside the target city or radius, and store or reply with the result. This tutorial builds an upload endpoint with a geo gate: a partner posts a file of locations, and only the rows relevant to your city ever reach storage. The same shape covers delivery zones, service areas and region-scoped imports. It assumes a registered endpoint; see [Build Your First API](https://docs.rual.nl/cluster/api-quickstart). The geo side of this page runs offline against datasets on the node; background in [Places and Geo Search](https://docs.rual.nl/blueprints/places). ## 1. Upload and Parse [`function_get_file_from_form`](https://docs.rual.nl/block-types/files/function_get_file_from_form) (get) takes the uploaded file off the multipart form by field name. Parse it into an array of row objects: - [`file_parse_csv`](https://docs.rual.nl/block-types/files/file_parse_csv) (parse CSV) detects the separator, reads headers, and outputs one object per row. Its `error` pin catches files that are not CSV at all. - [`file_parse_xlsx`](https://docs.rual.nl/block-types/files/file_parse_xlsx) (parse XLSX) and [`file_parse_xlsx_to_array`](https://docs.rual.nl/block-types/files/file_parse_xlsx_to_array) do the same for Excel uploads. Only accepting the file for later? [`function_save_file_from_form`](https://docs.rual.nl/block-types/files/function_save_file_from_form) (save) stores it as an asset instead of parsing it in the request. ## 2. The City Filter Two blocks answer "is this row inside my area", depending on whether the area is a city or a drawn radius: An uploaded file becomes rows: [get](https://docs.rual.nl/block-types/files/function_get_file_from_form) takes the file from the form, [parse CSV](https://docs.rual.nl/block-types/files/file_parse_csv) turns it into an array of objects, and [filter by geo distance](https://docs.rual.nl/block-types/array/array_filter_geo_distance) keeps only the rows whose location lies within their own radius of [from city](https://docs.rual.nl/block-types/geopoint/geo_point_from_city) (here Breda). [array length](https://docs.rual.nl/block-types/array/array_length) counts what survived. [![Studio canvas example for the parse CSV block: upload, parse, filter by city.](https://docs.rual.nl/canvas-examples/file_parse_csv.png)](https://docs.rual.nl/canvas-examples/file_parse_csv.png?v=3) On the canvas: [`geo_point_from_city`](https://docs.rual.nl/block-types/geopoint/geo_point_from_city) (from city) resolves the target city to its centre point, and [`array_filter_geo_distance`](https://docs.rual.nl/block-types/array%2Cgeopoint/array_filter_geo_distance) (filter by geo distance) keeps the rows whose own location field lies within reach. The block reads the location and the allowed distance from each row itself: `location_key` names the field holding the row's geo point, `distance_key` the field holding its radius. Rows carry their own reach, so a single pass handles "within 5 km" for one row and "within 500 m" for the next. What does not match comes out on `out_of_range`, so you can report or log the rejects instead of dropping them silently. Rows that store plain lat/lng numbers instead of a geo point need one conversion first: [`geo_point_lat_lng_to_point`](https://docs.rual.nl/block-types/geopoint/geo_point_lat_lng_to_point) (create). For a fixed radius applied to every row, test each row's point with [`condition_geo_is_point_in_circle`](https://docs.rual.nl/block-types/condition%2Cgeopoint/condition_geo_is_point_in_circle) (point in radius) inside an iterator, or filter at the storage level with [`query_bool_geo_distance`](https://docs.rual.nl/block-types/query/query_bool_geo_distance) so documents outside the area are never read at all. ## 3. Validate, Enrich, Store Before the surviving rows reach storage: - Check each row's point with [`geo_point_validate`](https://docs.rual.nl/block-types/geopoint/geo_point_validate) (is valid): uploads regularly contain empty or swapped coordinates, and one bad row should not poison the batch. - Enrich what you keep with [`geo_point_reverse_geocode`](https://docs.rual.nl/block-types/geopoint/geo_point_reverse_geocode) (reverse geocode), which resolves the point to street, city, province and country. Writing those onto the document with [`mutations_set_bp_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_set_bp_field_multiple) (set fields) makes the rows searchable by city name later, a combination production blueprints use directly on the upload path. - Write the batch with [`function_upsert_document_from_mutations`](https://docs.rual.nl/block-types/storage/function_upsert_document_from_mutations) (upsert document) inside a [`function_foreach`](https://docs.rual.nl/block-types/foreach/function_foreach) (foreach) loop: re-uploading the same file then updates rows instead of duplicating them. Reply with a small summary object: rows received, kept, rejected. Build it with [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields) (new object) fed by [`array_length`](https://docs.rual.nl/block-types/array/array_length) (array length) on both output arrays, and send it with [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) (reply in JSON), the response backbone from [Common Blueprint Patterns](https://docs.rual.nl/blueprints/common-patterns#api-response). ## Notes From Practice - Large files do not belong in the request: parse, reply 202, and filter and store through [`function_custom_execute_from_queue`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute_from_queue) (execute in queue). See [Queue](https://docs.rual.nl/blueprints/queue). - The geo blocks are offline lookups, so filtering ten thousand rows costs no external calls and no meaningful time. - Scope the endpoint and rate limit it; an open upload endpoint is an invitation. See [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control). - Branch on the parse block's `error` output before touching the array: replying 400 with "not a CSV" beats storing nothing and replying 200. - [Places and Geo Search](https://docs.rual.nl/blueprints/places): The offline dataset behind the geo filter blocks, and the search blocks built on it. - [File Upload & Processing](https://docs.rual.nl/tutorials/file-upload-processing): Forms, assets and background pipelines for uploads in general. - [Common Blueprint Patterns](https://docs.rual.nl/blueprints/common-patterns): The response backbone and error-handling shapes used here. ## Frequently asked **How do I accept a CSV upload through a RUAL API?** Take the file off the multipart form with function_get_file_from_form, then parse it with file_parse_csv into an array of row objects. The parse block detects the separator and reports files that are not CSV on its error pin. **How do I keep only uploaded rows inside a city in RUAL?** Resolve the city with geo_point_from_city and filter the parsed rows with array_filter_geo_distance, which reads each row's location and allowed radius from fields you name. Rows that fall outside come out on the out_of_range pin, and all of it runs offline against datasets on the node. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Building an API with Locking · RUAL Documentation" description: "Claim a lock around read-modify-write sections and always release it." canonical: https://docs.rual.nl/tutorials/api-locking language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Building an API with Locking Protect read-modify-write endpoints from concurrent callers: claim a lock around the critical section, and make sure it is always released, on success and on failure. Two requests updating the same document at the same moment lose one of the updates: both read, both compute, both write, and the second write overwrites the first. This tutorial wraps that read-modify-write section in a lock, so concurrent callers queue up instead of colliding. It assumes a registered endpoint and document updates; see [Build Your First API](https://docs.rual.nl/cluster/api-quickstart) and [Storages](https://docs.rual.nl/blueprints/storage#updating-documents). Background on when the platform locks for you (and when it does not) is in [Locking and Concurrency](https://docs.rual.nl/blueprints/locking). ## 1. The Lock Three blocks, one job each: | Block | Does what | | --- | --- | | [`function_claim_lock_wait`](https://docs.rual.nl/block-types/locking/function_claim_lock_wait) (lock or wait) | Claims the lock for a `key`, waiting until the current holder releases it. The right default for an API: every caller gets its turn. | | [`function_claim_lock_v2`](https://docs.rual.nl/block-types/locking/function_claim_lock_v2) (claim lock) | Tries once and reports through its `condition` out pin whether it got the lock. Use it when a failed attempt should answer "try again later" instead of waiting, for example a sync endpoint where a retry is cheaper than a queue. | | [`function_claim_unlock`](https://docs.rual.nl/block-types/locking/function_claim_unlock) (free lock) | Releases the key. Must run on every path, including the failures. | The `key` names what is being protected. Build it from the identifier of the thing being updated with [`value_concatenate`](https://docs.rual.nl/block-types/value/value_concatenate) (concatenate), for example `member-` plus the member guid: one lock per document, so updates to different members never wait on each other. The `expiry` (seconds) is the safety net that frees a key whose holder crashed before unlocking. ## 2. The Critical Section Between claim and release sits the work that must not interleave: read the document, compute, write it back. The critical-section shape: a [function trigger](https://docs.rual.nl/block-types/globals/trigger_custom_function) runs the flow, [lock or wait](https://docs.rual.nl/block-types/locking/function_claim_lock_wait) claims the key (waiting when another request holds it), the document is read with [get document](https://docs.rual.nl/block-types/storage/function_get_document) and written with [update document](https://docs.rual.nl/block-types/storage/function_update_document_mutations), and a [funnel](https://docs.rual.nl/block-types/funnels/funnel) merges the paths so [free lock](https://docs.rual.nl/block-types/locking/function_claim_unlock) always releases the key. [![Studio canvas example for the lock or wait block: lock around a document update.](https://docs.rual.nl/canvas-examples/function_claim_lock_wait.png)](https://docs.rual.nl/canvas-examples/function_claim_lock_wait.png?v=3) On the canvas: after the lock is claimed, [`function_get_document`](https://docs.rual.nl/block-types/storage/function_get_document) (get document) reads the current state, [`mutations_set_bp_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_set_bp_field_multiple) (set fields) builds the changes, and [`function_update_document_mutations`](https://docs.rual.nl/block-types/storage/function_update_document_mutations) (update document) writes them. The [`funnel`](https://docs.rual.nl/block-types/funnels/funnel) is the load-bearing detail: it merges the exit paths (success, validation failure, not found) into one flow that always reaches the unlock block. Every path you add later gets the same treatment: point its end at the funnel, never straight out of the function. ## Notes From Practice - Keep the critical section small. Everything slow (external calls, file work, email) belongs outside the lock; every second inside it is a second other callers wait. - Size the `expiry` to the worst realistic run of the section, not the average. Too short and a slow run loses its lock mid-write; too long and a crashed holder blocks everyone until timeout. - A lock is not a transaction. When the write itself must be atomic, use the document's own update blocks, which lock themselves per document (see [Locking and Concurrency](https://docs.rual.nl/blueprints/locking)); claim your own lock for invariants that span the read-modify-write as a whole, like "only one active subscription per member". - Waiting callers still count against the request timeout. When waits can be long, hand the work to [`function_custom_execute_from_queue`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute_from_queue) (execute in queue) with a `unique_id` per key instead; the queue serializes for you and the API replies immediately. See [Queue](https://docs.rual.nl/blueprints/queue). - [Locking and Concurrency](https://docs.rual.nl/blueprints/locking): Which blocks lock themselves, and when to claim a lock manually. - [Queue](https://docs.rual.nl/blueprints/queue): Serialize work per key without holding an HTTP request open. - [Common Blueprint Patterns](https://docs.rual.nl/blueprints/common-patterns): The endpoint and error-handling shapes around the lock. ## Frequently asked **How do I lock a critical section in a RUAL API?** Claim a key with function_claim_lock_wait before the read-modify-write section and release it with function_claim_unlock afterwards, merging every exit path through a funnel so the unlock always runs. Build the key from the updated document's identifier so different documents never wait on each other. **What happens when a RUAL blueprint crashes while holding a lock?** The lock's expiry frees the key after the given number of seconds, so a crashed holder cannot block other callers forever. Size the expiry to the worst realistic run of the critical section. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Local Home Automation · RUAL Documentation" description: "Automate the building around a Nano: which radio to buy, the daemon stack, adoption and the blocks." canonical: https://docs.rual.nl/tutorials/local-home-automation language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Local Home Automation Zigbee, Thread/Matter, Z-Wave, Philips Hue and Daikin on a RUAL Core Nano: which radio to buy, the daemon stack around the node, adoption, and the blocks that drive the devices. A RUAL Core Nano can automate the building it sits in: lights, sensors, switches and climate, driven by the same blueprints and the same document model as everything else on the node, with no vendor cloud in the loop. This tutorial is the whole setup on one box: which radio to buy, the daemons that sit between the node and the radios, how devices get adopted, and the blocks you build with. It assumes the node itself is already running; see [RUAL Core Nano](https://docs.rual.nl/architecture/core-nano) for the machine and the base configuration. ## Which Radio to Buy Zigbee, Thread and Z-Wave are three separate radios on three different frequency bands. Nothing bridges them, and no single dongle does all three. What you need depends entirely on what your devices speak: | Devices | Protocol | Radio needed | | --- | --- | --- | | Philips Hue bulbs | Zigbee (or via a Hue bridge over LAN) | Zigbee dongle, or nothing if you keep the bridge | | Aqara sensors, buttons, plugs | Zigbee (newer models: Thread/Matter) | Zigbee dongle | | IKEA Trådfri | Zigbee | Zigbee dongle | | Eve, newer Nanoleaf, newer Aqara | Thread (Matter) | Thread dongle + Matter Server | | Fibaro, Aeotec, Shelly Wave | Z-Wave | Z-Wave dongle | | Daikin air conditioning | WiFi / LAN | None, it is an IP device | #### Buy First: Home Assistant Connect ZBT-2 Silicon Labs MG24 based, and the right first purchase for almost everyone: Hue and Aqara are the two ecosystems most people start from, and both are Zigbee. Flash it with Zigbee firmware, run [Zigbee2MQTT](https://www.zigbee2mqtt.io) against it, and RUAL sees every device on the mesh. ## One radio at a time The ZBT-2 hardware can do Zigbee or Thread, but not both simultaneously. Home Assistant tested that ("multiprotocol" / MultiPAN) thoroughly on the previous ZBT-1, found it caused device instability, and explicitly does not support it. They recommend dedicating a dongle per protocol. So if you want Zigbee *and* Thread, buy two ZBT-2s. #### Buy Second, If You Have Z-Wave Devices: Home Assistant Connect ZWA-2 Z-Wave 800 series (Silicon Labs ZG23), with Z-Wave Long Range support and a foot-long antenna tuned for the sub-GHz band. Runs alongside a ZBT-2 with no conflict whatsoever: different frequency band (868 MHz in the EU vs 2.4 GHz), different USB device, different daemon. Note that none of Hue, Aqara or Daikin are Z-Wave. If you do not already own Z-Wave hardware, this is not your first purchase. #### Running Both Two dongles on one machine is the ordinary configuration, not an exotic one. The M2 Mac Mini has exactly two USB-A ports, so two dongles fit natively. The M4 Mac Mini has no USB-A ports (front USB-C, rear Thunderbolt), so there they hang off a powered USB-C hub. Either way, RUAL namespaces every device by its adapter (`zigbee:0x00158d...`, `zwave:12`, `matter:4`), so nothing collides and one device picker shows them all. ## Use the extension cables Both dongles ship with one, and a dongle plugged straight into the machine is the single most common cause of a Zigbee mesh that "works but drops devices". USB 3.0 ports and NVMe SSDs emit broadband noise right across 2.4 GHz, and a Zigbee coordinator on the back of the machine sits inside that. Half a metre of cable, away from the case, fixes it. On a hub the same rule applies: dongles on their cables, not packed into adjacent ports. The ZWA-2's antenna wants to be vertical, and has an accelerometer specifically to nag you about that. ## The Stack Around the Node RUAL does not drive the radios itself. A native Zigbee stack means owning EZSP/Z-Stack framing, the ZCL cluster library, mesh routing, OTA updates and a device quirks database: years of work whose output would be a worse Zigbee2MQTT. So the node sits behind the reference daemon for each protocol: | Protocol | Daemon | How RUAL reaches it | | --- | --- | --- | | Zigbee | Zigbee2MQTT | MQTT | | Z-Wave | Z-Wave JS UI | MQTT (gateway type must be **ValueID topics**) | | Thread / Matter | Matter Server + a Thread Border Router | WebSocket | | Philips Hue | the bridge itself | HTTPS + event stream, on the LAN | | Daikin | the BRP WiFi adapter itself | HTTP, on the LAN | This is also where the device knowledge base comes from. Zigbee2MQTT publishes a complete machine-readable description of every device on your mesh, covering every attribute it reports with its type, unit and range. Roughly 4,000 devices, maintained by people who own the hardware. RUAL reads that directly, so pairing a sensor released after your RUAL version still gives you a fully described device. There is no device list in RUAL to update. You will also need **Redis** and an **MQTT broker** (Mosquitto). Both are small. On a Mac Mini, running the daemons in Docker Desktop and the node natively is the least fuss. ## Configuration > **Off by default** A node without `[homeassist] enabled = true` starts no radio goroutine, creates no device store, and every home-automation block reports `HOMEASSIST_NOT_AVAILABLE`, so a cloud node pays nothing for these blocks existing. Turn it on in the node configuration: ```toml [homeassist] enabled = true entity = 1 [homeassist.zigbee] broker = "tcp://127.0.0.1:1883" base_topic = "zigbee2mqtt" ``` ## Discovery and Adoption Devices appear on their own. A Zigbee coordinator hears everything ever paired to its network, and the node records each one as it reports. But a discovered device is **not** in service yet: it fires no triggers and refuses every command until you **adopt** it. That gate is the whole reason this works. Without it, "when any sensor activates" means "when anything within thirty metres twitches", including the sensor somebody paired to test three years ago and the devices a second-hand Z-Wave stick inherited from its previous owner. Adoption is what turns a radio's earshot into a system you can reason about. The onboarding loop: - Open the radio for joining, with the `pairing open` block or `POST /_system/homeassist/pair`. The window is bounded by `pairing_window`; a permanently joinable coordinator is how a mesh acquires devices nobody added. - Put the device into pairing mode (usually a long press). - It appears as discovered. The `device discovered` trigger fires. It is the one trigger that fires for unadopted devices, because noticing them is its purpose. - Adopt it, and give it a name and a room. From that moment its triggers fire and blocks can command it. You can automate step 4: wire `device discovered` straight into `device adopt`. For a bench mesh, `auto_adopt = true` skips it entirely, but leave that off anywhere real. ## Blocks Every device, whatever protocol it speaks, is described in one vocabulary. A motion sensor is `occupancy` whether it is an Aqara on Zigbee, a Fibaro on Z-Wave or an Eve on Thread, so a flow written against one works against the others. Two vendor quirks are corrected at the boundary rather than left to you: **contact sensors read `true` when the door is OPEN** (Zigbee and Matter both report the opposite on the wire), and **colour temperature is in kelvin**, not mireds. #### Triggers: These Start a Flow | Block | Fires when | | --- | --- | | `sensor activated` / `sensor deactivated` | Any sensor turns on or off. Narrow it with the sensor type pin, or leave it open for the whole house. | | `presence detected` / `presence cleared` | Occupancy specifically. | | `contact opened` / `contact closed` | A door or window. | | `light turned on` / `light turned off` | A light changes state, including from a wall switch or the vendor's own app. | | `switch turned on` / `switch turned off` | A plug or relay. | | `button pressed` | A button or scene switch. The value pin says single, double or hold. | | `water leak detected`, `smoke detected` | Alarm sensors. | | `temperature changed`, `humidity changed` | A reading changes, or crosses a threshold you set. | | `battery low`, `device unavailable` | Device health. | | `device discovered`, `device adopted` | The fleet changes. | | `device state changed` | Anything at all. High volume, so use a specific trigger where one exists. | Transition triggers fire on the **change**, not on every report. A motion sensor re-announcing `occupancy: true` every thirty seconds while somebody stands in the hallway fires your flow once, on entry. Threshold triggers likewise fire on the **crossing**: "above 25 °C" fires once when the room passes 25, not every minute for the rest of the afternoon. #### Actions | Block | Does | | --- | --- | | `light on` | Turns a light on, optionally setting brightness, kelvin and colour in the same command, so the lamp transitions once instead of visibly stepping through full white first. | | `light off`, `light toggle`, `light dim` | The rest of the lighting set. | | `switch on` / `switch off` | Plugs and relays. | | `climate set` | Mode, target temperature and fan speed on an air conditioner or thermostat. | | `device set` | Any capability by name: blind position, lock state, anything the specific blocks do not cover. | | `get all devices`, `get active sensors`, `get lights that are on`, `get rooms` | Query the registry. These read memory, make no network calls, and are cheap enough to call in a loop. | | `get device`, `get device state` | One device, or one capability of one device. | | `device adopt`, `device rename`, `device forget`, `pairing open` | Fleet management from a flow. | | `get status` | Which radios are connected. A bridge that has gone away otherwise looks exactly like a quiet house. | ## History and Energy The node records a bounded history of what its devices do, and the word doing the work is *bounded*. A dozen sensors produce tens of reports a minute. A motion sensor re-announces while you are in the room, a power meter reports every ten seconds, every Zigbee message carries a link-quality stamp. Storing each one would be millions of rows a month whose information content is nearly zero. So nothing stores what a device *said*. It stores what *changed*: | Kind | Examples | Stored | Charted as | | --- | --- | --- | --- | | Alarm | occupancy, contact, leak, smoke | On transitions. A door open for eight hours is two rows, not four hundred and eighty. | Timeline; summarised as how long it was true | | Measurement | temperature, humidity, power draw, lux | Past a per-capability deadband, plus a heartbeat. A thermometer drifting 0.02 °C stores nothing; a room warming up stores a curve. | Line; min / average / max | | Meter | energy | At a fixed interval, because a lifetime counter changes on every report. | Bars of consumption per window, never the raw total | | Event | button presses | Every one, because every occurrence is the data. | not charted | | State | on/off, brightness, mode | On change. | Steps | The heartbeat is worth understanding: without it, a deadband makes a stable sensor indistinguishable from a dead one, because both produce no rows. With it, a flat line is visibly flat and a gap is visibly a gap. Two things this makes possible. The `get device history` block returns raw readings and, on request, aggregated windows, so a flow can answer "was anyone in the hall between two and four" or "how much did the boiler use last week" without an external time-series database. And RUAL Studio's **Devices → Home Automation** screen charts it: per-capability history with 1h/24h/7d/30d ranges, and an energy panel showing consumption per device. Retention defaults to 90 days and readings are pruned hourly. Set `[homeassist.analytics] enabled = false` to record nothing; the history block then reports `recording = false`, which is deliberately distinguishable from an empty result. ## What to Know Before You Commit - **Device state is not persisted across restarts.** Which devices exist, their names, rooms and adoption all survive; what they were *doing* does not. For the first few seconds after a restart, "which lights are on" honestly answers "I have not been told yet" rather than guessing. Blocks report `state_known` so a flow can tell the difference. - **Triggers are not durable.** Dispatch is in-process, so a restart drops anything in flight. That is deliberate: replaying a motion event from before the restart would light a room somebody left ten minutes ago. - **Smoke and leak triggers are a supplement, not a certified alarm.** They depend on a mesh, a broker and a node all being up. - **Daikin is polled, not pushed.** Its local API has no event channel, so `poll_interval` (default 30s) is the resolution of every climate trigger. The newer BRP069C4x "Onecta" firmware uses a different protocol and is not supported yet. - **Matter endpoints collapse.** A Matter device with two endpoints, such as a two-gang switch, currently appears as one device. - **History is not a metering-grade record.** It is sampled by design, so it answers trends and totals well and forensic questions poorly. If you need every reading, for billing or for compliance, write them to a storage from a flow instead. ## A First Flow Hallway light on when someone walks in, off when they leave, but only after dark: - `presence detected`, room `Hallway` → `get device state` on the hallway sensor, capability `illuminance`. - `if` the reading is below 30 lux → `light on`, brightness 40, kelvin 2700. - Separately: `presence cleared`, room `Hallway` → `light off`. Both triggers fire on the transition, so neither re-runs while somebody stands still. That is four blocks, and it is the shape most home automation takes. ## Next Steps - [Home Automation](https://docs.rual.nl/home-automation/introduction): The full section: the setup runbook, every trigger and action block, worked flows, history and troubleshooting. - [RUAL Core Nano](https://docs.rual.nl/architecture/core-nano): The deployment shape this tutorial runs on. - [All Blocks](https://docs.rual.nl/block-types): The full catalog, including every home-automation block. - [Repeating Events](https://docs.rual.nl/blueprints/repeating-events): Automation on a clock instead of a sensor. ## Frequently asked **Which radio do I need for home automation on a RUAL Core Nano?** It depends on what your devices speak: Zigbee devices need a Zigbee dongle, Thread and Matter devices need a Thread dongle with Matter Server, and Z-Wave devices need a Z-Wave dongle. The Home Assistant Connect ZBT-2 is the recommended first purchase, and one dongle does Zigbee or Thread, not both at once. **Does RUAL drive Zigbee or Z-Wave radios itself?** No. The node sits behind the reference daemon for each protocol: Zigbee2MQTT for Zigbee, Z-Wave JS UI for Z-Wave, and Matter Server for Thread and Matter, reached over MQTT or WebSocket. You also need Redis and an MQTT broker such as Mosquitto. **Why do I have to adopt devices before RUAL automates them?** A discovered device fires no triggers and refuses every command until you adopt it. A coordinator hears everything ever paired to its network, so adoption is what turns a radio's earshot into a system you can reason about. **Does RUAL store every reading a smart home sensor sends?** No, it stores what changed: alarms and states on transitions, measurements past a per-capability deadband plus a heartbeat, meters at a fixed interval, and button presses individually. Retention defaults to 90 days with hourly pruning. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Home Automation · RUAL Documentation" description: "Running the building the node sits in: Zigbee, Thread/Matter, Z-Wave, Philips Hue and Daikin devices, adopted into one vocabulary and automated by blueprints, with no vendor cloud in the loop." canonical: https://docs.rual.nl/home-automation language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Home Automation Running the building the node sits in: Zigbee, Thread/Matter, Z-Wave, Philips Hue and Daikin devices, adopted into one vocabulary and automated by blueprints, with no vendor cloud in the loop. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Introduction to Local Home Automation · RUAL Documentation" description: "What local home automation means here: the node as the gateway, one capability vocabulary across four radios, and what you need to own." canonical: https://docs.rual.nl/home-automation/introduction language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Local Home Automation A RUAL node is the gateway itself, not a client of somebody else's cloud. Zigbee, Thread/Matter, Z-Wave, Philips Hue and Daikin devices become one vocabulary that blueprints automate. A RUAL node can run the building it sits in. Lights, sensors, buttons, plugs, blinds, locks and climate are adopted into the same document model as everything else on the node, and driven by the same blueprints you already write for pages and endpoints. Nothing goes through a vendor cloud, and nothing stops working when the internet does. This section is the working reference for that. If you have not set the hardware up yet, start at [Setting Up a RUAL Nano](https://docs.rual.nl/home-automation/setting-up). If you want the shorter tour first, read the [Local Home Automation tutorial](https://docs.rual.nl/tutorials/local-home-automation); this section goes considerably deeper. ## The Node Is the Gateway The ordinary way to reach a smart device from software is to call the vendor's cloud API. Your flow talks to a server in another country, that server talks to a bridge in your hallway, and the bridge talks to a lamp two metres away. That is a round trip measured in hundreds of milliseconds on a good day, it needs an account and a rate limit, and it fails whenever any hop between you and the vendor does. Here the node is the thing the devices talk to. Radios are attached to the machine, the protocol daemons run beside the node, and a motion event reaches your flow over a loopback socket. The distinction shows up in three places you will notice: | | Vendor cloud integration | Local, on the node | | --- | --- | --- | | A motion event reaching a flow | Sensor to bridge to vendor cloud to your cluster | Sensor to radio to node, in process | | Internet down | Nothing works | Everything works | | Vendor retires the API | Your automation stops | Unaffected: the radio protocol has not changed | | Who can see your floor plan | The vendor, by construction | Nobody outside the LAN | Aqara is the clearest case. Aqara devices *are* Zigbee devices. Reached through Aqara's cloud API, turning on a lamp is a request to Germany and back; paired to a coordinator on your own machine, they are just devices on your mesh. That is why the older `aqaraconnection_*` blocks are deprecated in favour of these. ## What the Node Does Not Do RUAL Core does not drive the radios itself, and that is a deliberate line. A native Zigbee stack means owning EZSP and Z-Stack framing, the ZCL cluster library, mesh routing, OTA updates and a device quirks database: years of work whose output would be a worse Zigbee2MQTT. So the node sits behind the reference daemon for each protocol and translates. | Protocol | Runs beside the node | The node reaches it over | | --- | --- | --- | | Zigbee | Zigbee2MQTT, against a USB coordinator | MQTT | | Z-Wave | Z-Wave JS UI, against a Z-Wave stick | MQTT, gateway type **ValueID topics** | | Thread and Matter | Matter Server, plus a Thread Border Router for Thread devices | WebSocket | | Philips Hue | the bridge you already own | HTTPS and its event stream, on the LAN | | Daikin | the BRP WiFi adapter in the unit | HTTP, on the LAN, polled | | UniFi Access and Protect | the UniFi console you already own | Ubiquiti's integration APIs and their event streams, on the LAN | The payoff is the device knowledge base. Zigbee2MQTT publishes a complete machine-readable description of every device on your mesh: every attribute it reports, with type, unit and range, for roughly 4,000 devices, maintained by people who own the hardware. The node reads that directly, so pairing a sensor released after your RUAL version still gives you a fully described device. There is no device list inside RUAL to update. Hue is a separate adapter from Zigbee even though a Hue bridge *is* a Zigbee coordinator, because the two are different deployments. If a household already has a bridge, its bulbs are paired to that bridge, and re-pairing them to your own stick breaks the Hue app, the dimmer switches and the scenes the household already uses. Talking to the bridge leaves all of that working. A bulb paired directly to your own coordinator arrives through the Zigbee adapter instead, and neither path knows about the other. ## One Vocabulary, Whatever the Radio Four vendors describe the same physical event four different ways. A motion sensor is `occupancy: true` on Zigbee2MQTT, a Notification command class carrying "Home Security, Motion sensor status" on Z-Wave JS, `motion.motion: true` on a Hue bridge, and bit 0 of an OccupancySensing bitmap on Matter. An automation author who has to know which one they own has not been given an automation platform, they have been given a protocol decoder. So every adapter funnels into one closed set of **capabilities**, and a flow is written against those. A `sensor activated` trigger behaves identically whether the sensor is an Aqara on Zigbee, a Fibaro on Z-Wave or an Eve on Thread, because by the time the trigger sees it, all of them are `occupancy = true`. The set is deliberately small and closed. A capability earns its place by being something an automation would branch on, not by being something a radio happens to transmit. Anything not listed here still survives on the device's `raw` map, so it is readable without being promoted into the vocabulary. #### Binary Capabilities **True always means the active, noteworthy state.** That single rule is why a blueprint never has to ask whether `true` means open or closed. | Capability | True means | Counts as a sensor | | --- | --- | --- | | `occupancy` | Someone is there | Yes | | `contact` | The door or window is **open** | Yes | | `water_leak` | Water is present | Yes | | `smoke` | Smoke detected | Yes | | `gas` | Gas or carbon monoxide detected | Yes | | `vibration` | Moved, tilted or dropped | Yes | | `tamper` | The anti-tamper switch opened | No, device health | | `power` | The bulb, plug or relay is on | No, it has its own triggers | | `battery_low` | The device raised its own low-battery flag | No, device health | The "counts as a sensor" column is what `sensor activated` and `get active sensors` mean by a sensor: the binary capabilities that describe the *world*, minus the ones that describe the *device*. `power` is excluded because a bulb turning on is not a sensor activating, and folding it in would make every whole-house sensor flow fire on its own lighting changes and livelock. `tamper` and `battery_low` are excluded from the other direction: an author asking about presence does not want a flat battery in the same stream. #### Numeric Capabilities | Capability | Unit | Notes | | --- | --- | --- | | `temperature` | °C | Ambient reading | | `target_temperature` | °C | The setpoint somebody chose | | `humidity` | % | Relative | | `pressure` | hPa | Atmospheric | | `illuminance` | lux | Hue and Matter both report a log scale on the wire; converted here | | `brightness` | 0 to 100 % | Vendors use 0 to 254 or 0 to 99; adapters scale | | `color_temp` | **kelvin** | Not mireds. 2200 candle, 2700 warm white, 4000 neutral, 6500 daylight | | `position` | 0 to 100 % | Blinds and valves, percent open | | `power_watts` | W | Instantaneous draw | | `energy_kwh` | kWh | Cumulative lifetime meter | | `co2` | ppm | | | `voc` | ppb | | | `pm25` | µg/m³ | | | `battery` | % | Remaining charge | | `link_quality` | 0 to 255 | The radio's own signal metric | #### Text and Enum Capabilities | Capability | Values | | --- | --- | | `color_hex` | `#rrggbb` | | `hvac_mode` | `off`, `auto`, `cool`, `heat`, `dry`, `fan` | | `fan_mode` | `auto`, `silent`, `low`, `medium`, `high`, `turbo` | | `lock_state` | `locked`, `unlocked`, `jammed` | | `action` | A momentary button pulse: `single`, `double`, `hold`, `release`, and whatever else the device emits | #### Access Control and Cameras These arrive with the UniFi adapter and are shaped for a different kind of device. A door controller is a lock, a position sensor, a card reader and a doorbell at once, and "the front door" is one thing people automate rather than four. | Capability | Carries | Behaves like | | --- | --- | --- | | `door_position` | true means the door or gate is **open**. The physical position, not the lock: a door can be unlocked and shut. | An alarm: stored on transitions, charted as a timeline | | `alarm` | true means the security controller's own alarm is active. Distinct from the smoke, gas and leak sensors, which are individual detectors. | An alarm | | `doorbell` | Somebody rang. The value is the moment it happened. | An event: every occurrence is recorded | | `access_entry` | An entry decision, as an object: `actor`, `method`, `granted`, `door`. Method is `nfc`, `pin`, `wave`, `mobile`, `touchpass` or `manual`. | An event | | `nfc` | A card or fob was presented, whether or not it was accepted. The value is the token or actor. | An event | | `motion_event` | A camera detected motion. The value is the moment it happened. | An event | `nfc` is separate from `access_entry` on purpose: "a card was swiped" and "entry was granted" are different automations, because a swipe may be refused. `motion_event` is separate from `occupancy` for the same kind of reason. A camera reports motion as discrete detections; a PIR or radar sensor reports occupancy as a state that stays true while somebody is there. Use `presence detected` for the second and `camera motion detected` for the first. #### The Two Corrections Worth Knowing Two vendor quirks are fixed at the adapter boundary rather than left for you to discover: ## Contact sensors read `true` when the door is OPEN Zigbee's contact attribute is true when the magnet is present, which is when the door is *closed*, and Matter's BooleanState is the same way round. Both are inverted on the way in. This single inversion is the most common source of backwards door automation in every home automation system there is, so it is corrected once, at the boundary: `contact = true` means open, here and everywhere above here. ## Colour temperature is kelvin, not mireds Zigbee, Hue and Matter all speak mireds on the wire, so adapters convert. "Warm white is 2700" is knowledge an automation author has; "warm white is 370" is not. ## Device Types Are Derived, Not Declared Every device is classified into one of ten types, and the classification comes from the capabilities it exposes rather than from what the vendor calls itself. Vendor device classes are inconsistent across ecosystems and frequently wrong: Zigbee plugs that meter power routinely announce themselves as lights. | Type | Recognised by | | --- | --- | | `climate` | `hvac_mode` or `target_temperature` | | `door` | `doorbell`, `access_entry`, `nfc` or `door_position` | | `camera` | `motion_event` | | `lock` | `lock_state` | | `cover` | `position` | | `light` | `brightness`, `color_temp` or `color_hex` | | `sensor` | Any sensor capability, or any environment reading | | `button` | `action` | | `switch` | `power`, with none of the above | | `unknown` | Nothing recognisable yet | The order in that table is the order the checks run, and it is not alphabetical. Capabilities overlap: a Hue bulb has `power` and `brightness`, a metering plug has `power` and `power_watts`. Testing brightness before plain power is what keeps the plug out of the lights list, and testing climate before either is what keeps a thermostat with a display backlight from being classified as a lamp. Access control sits near the top for the same reason. A UniFi door has a lock state *and* a door-position sensor *and* usually a reader and a bell, so any access-control capability wins and the whole thing classifies as one `door` rather than as a lock, a sensor and a button. Type matters more than it looks. `light turned on` fires only for devices classified as lights, `switch turned on` only for switches, and the studio's device picker offers only the right type per block. ## Nothing Is In Service Until You Adopt It Devices appear on their own: a coordinator hears everything ever paired to its network, and the node records each one as it reports. But a discovered device fires no triggers and refuses every command until you **adopt** it. That gate is what makes the rest reasonable. Without it, "when any sensor activates" means "when anything within thirty metres twitches", including the sensor somebody paired to test three years ago and whatever a second-hand Z-Wave stick inherited from its previous owner. The full onboarding loop is on [Discovering and Adopting Devices](https://docs.rual.nl/home-automation/devices). ## What You Need to Own - **A machine.** A RUAL Core Nano: one box in the building, wired, on an SSD. 16 GB of RAM is the comfortable number once the radio daemons are running beside the node. See [RUAL Core Nano](https://docs.rual.nl/architecture/core-nano) for the specifications and the reference builds. - **A radio per protocol.** Zigbee, Thread and Z-Wave are three radios on different frequency bands. Nothing bridges them and no single dongle does all three at once. The [tutorial](https://docs.rual.nl/tutorials/local-home-automation) covers which to buy. - **The daemons.** Redis and an MQTT broker for the node itself, plus Zigbee2MQTT, Z-Wave JS UI or Matter Server depending on what you own. [Setting Up a RUAL Nano](https://docs.rual.nl/home-automation/setting-up) is the runbook. - **The devices.** Hue, Aqara, IKEA, Eve, Fibaro, Aeotec, Shelly Wave, a Daikin air conditioner and a UniFi console running Access or Protect are all covered. Anything Zigbee2MQTT supports works, which is most of what is sold. The blocks arrive with Core v15.0.0, and the feature is opt-in: > **Off by default** A node without `[homeassist] enabled = true` starts no radio goroutine, creates no device store, and every home-automation block reports `HOMEASSIST_NOT_AVAILABLE`, so a cloud node pays nothing for these blocks existing. ## Two adapters are not yet released The UniFi Access and Protect adapter, with its eight triggers and four door actions, and Daikin Onecta support are complete but not yet in a released Core build. The pages that use them say so where they appear. Everything else in this section describes the current Core release. ## What to Know Before You Commit - **Restored state is a belief, not an observation.** Last-known device state survives a restart (see below), but a restored value does not claim the device is reachable: `available` stays false and `last_seen` is untouched until the device actually reports. That is what keeps `get all devices` from announcing a fleet of confidently-reachable hardware seconds after boot. - **Triggers are not durable.** Device triggers dispatch in process rather than through the queue, which is what makes a light come on before you have finished walking through the door. The cost is that a restart drops anything in flight, and that is the right behaviour anyway: replaying a motion event from before the restart would light a room somebody left ten minutes ago. - **Smoke, gas and leak triggers are a supplement, not a certified alarm.** They depend on a mesh, a broker and a node all being up. - **Daikin is polled.** Its local API has no event channel, so `poll_interval` (30s by default) is the resolution of every climate trigger. - **Matter endpoints collapse.** A Matter device with two endpoints, such as a two-gang switch, currently appears as one device whose power state is whichever endpoint reported last. - **The UniFi adapter is not verified against hardware yet.** Its transport follows Ubiquiti's documented Access and Protect integration APIs, but the change that introduced it could not confirm the field decoding end to end against a live console. Treat the translated fields as best-effort; every payload is also carried untranslated on the device's `raw` map, so nothing is lost when a field name differs. - **Daikin Onecta temperatures are best-effort.** Power on and off is reliable; the temperature and mode decoding follows community reverse engineering, the hex encoding varies by model, and it is unverified against a live unit. The full raw `/dsiot` tree is on the device's `raw` map. - **History is sampled by design.** It answers trends and totals well and forensic questions poorly. See [Device History and Energy](https://docs.rual.nl/home-automation/history-and-energy). ## What a Restart Looks Like Worth understanding before you write a flow that runs at boot, because the naive design of this feature has a memorable failure mode: a window that was open all night announces itself as "just opened" at 03:00, the alarm flow runs, and the lights come on. Two mechanisms prevent that. **An unknown previous value seeds silently rather than firing.** When the node has never seen a capability before, an incoming reading is recorded without counting as a transition. The exceptions are the three where a missed event is worse than a repeated one: `smoke`, `gas` and `water_leak` still fire when they arrive active, so a node that comes back up to an alarm already sounding says so. Buttons also still fire, because a press is an event with no persistent state to seed. **Last-known state is persisted and restored before the adapters connect**, so the first real report has something to compare against and an unchanged re-announce stays unchanged. It lives in Redis rather than the document store, because it is hot, small, rewritten constantly and worthless once stale: the exact profile of a cache, and the exact opposite of the identity data. Writes coalesce per device on a two-second flush (one Zigbee message carrying four capabilities is one write), a clean shutdown flushes, and entries expire after 30 days so a node restored from a backup does not seed year-old beliefs. Momentary capabilities are excluded from the persisted map. Replaying the last button press as current state would make it look freshly pressed. ## Next Steps - [Setting Up a RUAL Nano](https://docs.rual.nl/home-automation/setting-up): The runbook: broker, Zigbee2MQTT against a ZBT-2, config.toml and first boot. - [Discovering and Adopting Devices](https://docs.rual.nl/home-automation/devices): Pairing, adoption, rooms and naming. - [Example Flows](https://docs.rual.nl/home-automation/example-flows): Nine worked automations, block by block. ## Frequently asked **Can RUAL control smart home devices without a cloud?** Yes. A RUAL node is the gateway itself: the radios are attached to the machine, the protocol daemons run beside the node, and a motion event reaches a blueprint over a loopback socket. Zigbee, Thread and Matter, Z-Wave, a Philips Hue bridge and Daikin air conditioning are all reached on the local network, so the automations keep working when the internet does not. **Does a RUAL flow have to know whether a sensor is Zigbee or Z-Wave?** No. Every adapter translates into one closed set of capabilities, so a motion sensor is occupancy whether it is an Aqara on Zigbee, a Fibaro on Z-Wave or an Eve on Thread. A flow written against one works against the others without an edit. **Does a RUAL contact sensor report true when the door is open or closed?** True means OPEN. Zigbee and Matter both report the opposite on the wire, because their contact attribute is true when the magnet is present, and RUAL inverts it at the adapter so that true always means the active, noteworthy state. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Setting Up a RUAL Nano · RUAL Documentation" description: "The runbook: Redis, an MQTT broker, Zigbee2MQTT against a ZBT-2, the config section, first boot and proving the radio is connected." canonical: https://docs.rual.nl/home-automation/setting-up language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Setting Up a RUAL Nano for Home Automation A runbook, in order: Redis, an MQTT broker, Zigbee2MQTT against a ZBT-2, the node's config.toml, and how to prove the radio is actually connected before you start writing flows. This page assumes a machine that already runs RUAL Core. The hardware choice, the minimum specifications and the growth path are on [RUAL Core Nano](https://docs.rual.nl/architecture/core-nano); which radio to buy, and why one dongle cannot do Zigbee and Thread at the same time, is in the [Local Home Automation tutorial](https://docs.rual.nl/tutorials/local-home-automation). Everything below is what happens after the box and the dongle are on the desk. The order matters. Each step verifies before the next one depends on it, because a home-automation stack that is wrong in the middle presents identically to one that is wrong at either end: no devices. ## What You Are Building Four processes, three of them not RUAL: | Process | Role | Reached over | | --- | --- | --- | | Redis | Cache and pub/sub the node already needs, plus last-known device state | `127.0.0.1:6379` | | Mosquitto | MQTT broker: the bus the radio daemons publish to | `127.0.0.1:1883` | | Zigbee2MQTT | Owns the USB coordinator, speaks Zigbee, publishes to the broker | the broker | | RUAL Core | Reads the broker, translates into capabilities, runs your flows | the broker | Add Z-Wave JS UI or Matter Server later and they join the same pattern: their own process, their own radio, publishing where the node can read them. ## Step 1: Redis Redis is not home-automation specific: the node uses it for its document cache and for the pub/sub channel that reloads the trigger index when you edit a blueprint. Home automation adds one job to it, holding last-known device state so a restart does not re-fire every already-active sensor. If your node already runs, Redis already runs, and you can skip to step 2. ``` brew install redis brew services start redis redis-cli ping # PONG ``` ``` sudo apt install redis-server sudo systemctl enable --now redis-server redis-cli ping # PONG ``` Bind it to localhost. It holds cached documents and nothing on it needs to be reachable from the network. ## Step 2: The MQTT Broker Mosquitto is the broker both MQTT adapters speak to. It needs a config file before it will listen on anything other than the loopback interface with no authentication, and for a single box that default is exactly what you want. ``` brew install mosquitto brew services start mosquitto ``` ``` sudo apt install mosquitto mosquitto-clients sudo systemctl enable --now mosquitto ``` Prove it works before anything depends on it. Two terminals: subscribe in one, publish in the other. ``` # terminal 1 mosquitto_sub -h 127.0.0.1 -t 'test/#' -v # terminal 2 mosquitto_pub -h 127.0.0.1 -t 'test/hello' -m 'it works' ``` If terminal 1 prints `test/hello it works`, the bus is up. If it does not, stop here: everything after this point fails the same way, silently, with no devices and no error. **If you set a username and password**, put the same pair in the node's `[homeassist.zigbee]` section and in Zigbee2MQTT's own config. An authenticated broker with one side unconfigured looks like a broken radio. ## Step 3: Zigbee2MQTT Against the ZBT-2 This is the step with the sharp edges. Zigbee2MQTT owns the USB coordinator, so it needs the actual device node, and that constrains how you may run it. #### Find the Serial Port Plug the ZBT-2 in, on its extension cable, away from the case. USB 3 ports and NVMe drives emit broadband noise right across 2.4 GHz, and a coordinator plugged directly into the back of a machine sits inside that. Then find the device: ``` # macOS ls /dev/cu.usbmodem* # Linux: use the by-id path, it survives a reboot and a re-plug ls -l /dev/serial/by-id/ ``` Use the `/dev/serial/by-id/...` path on Linux, never `/dev/ttyUSB0`. With two dongles attached, the numbered paths swap on reboot, and a Zigbee daemon that opens the Z-Wave stick fails in a way that takes an hour to recognise. #### Docker Cannot See the Dongle on a Mac Worth knowing before you follow a tutorial that assumes otherwise. Docker Desktop on macOS runs containers inside a Linux VM, and that VM has no USB passthrough: `--device /dev/cu.usbmodem...` does not work, and no flag makes it. So on a Mac Mini, Zigbee2MQTT runs **natively** under Node.js, while Redis and Mosquitto can live in Docker or under Homebrew, whichever you prefer. On Linux the ordinary Docker setup works, with the coordinator passed through by its by-id path. ``` # Node 20 or newer git clone --depth 1 https://github.com/Koenkk/zigbee2mqtt.git ~/zigbee2mqtt cd ~/zigbee2mqtt npm ci npm run build # writes data/configuration.yaml on first run npm start ``` #### configuration.yaml The parts that matter, in `data/configuration.yaml`: ``` mqtt: # This must match [homeassist.zigbee] base_topic on the node. base_topic: zigbee2mqtt server: mqtt://127.0.0.1:1883 # user: rual # password: ... serial: # macOS: /dev/cu.usbmodem... Linux: /dev/serial/by-id/... port: /dev/cu.usbmodem2101 # The ZBT-2 is Silicon Labs EmberZNet silicon. adapter: ember baudrate: 460800 # The web UI. Useful for pairing and for reading the map; not required by RUAL. frontend: port: 8099 advanced: # Leave permit_join off. RUAL opens the window when you ask it to, for a # bounded number of seconds. log_level: info permit_join: false ``` The adapter and baud rate are coordinator-specific. Those two lines are for the Home Assistant Connect ZBT-2; if you bought something else, check [the Zigbee2MQTT adapter list](https://www.zigbee2mqtt.io/guide/adapters/) rather than guessing, because a wrong adapter setting produces a daemon that starts and then never sees a device. ## Leave `permit_join` false A coordinator left permanently joinable is how a mesh acquires devices nobody added. Pairing is opened deliberately, for a bounded window, from RUAL: see [Discovering and Adopting Devices](https://docs.rual.nl/home-automation/devices). #### Prove the Radio Is Talking Start Zigbee2MQTT and watch the broker rather than the log. This is the single most useful diagnostic on the whole page, because it separates "the radio works" from "RUAL is configured correctly" once and for all: ``` mosquitto_sub -h 127.0.0.1 -t 'zigbee2mqtt/#' -v ``` Within a few seconds you should see `zigbee2mqtt/bridge/state` carrying `{"state":"online"}`, and `zigbee2mqtt/bridge/devices` carrying a JSON array. That array is the device knowledge base: every device on the mesh with its full `exposes` tree. On a fresh coordinator it is empty, which is correct. If `bridge/state` reports `offline`, the broker is fine and Zigbee2MQTT is not: it is either not running or it has lost the coordinator. That distinction matters enough that the node logs it separately; see [Troubleshooting](https://docs.rual.nl/home-automation/troubleshooting). ## Step 4: The Node's Configuration Home automation is off by default, and the switch is one key. Start from `configs/nano.example.toml` in the RUAL Core repository, which is this section plus the SQLite storage half. ``` [homeassist] enabled = true # Which tenant owns the devices. A radio is a physical building, and buildings # do not have tenants. On a single-entity node this may be omitted and falls # back to the one active entity; on a node serving several it is required. entity = 1 # Adopt every discovered device on sight. Leave this false anywhere real. auto_adopt = false # Ceiling on how long a pairing request may open a radio for joining. pairing_window = "2m" [homeassist.zigbee] broker = "tcp://127.0.0.1:1883" # Must match Zigbee2MQTT's own mqtt.base_topic. base_topic = "zigbee2mqtt" # username = "" # password = "" # Only needed when several RUAL nodes share one broker: two MQTT connections # with the same client id kick each other off forever. # client_id = "" ``` Four keys are worth understanding rather than copying. | Key | Default | Why it exists | | --- | --- | --- | | `entity` | the single active entity | Devices belong to one tenant. With exactly one entry in `general.active_entities` the node fills it in; on a node serving several it refuses to start rather than guess which tenant owns the building. | | `auto_adopt` | `false` | Adopts everything on sight. It exists for a bench mesh that is five minutes old, where adopting by hand between every pair is friction with no safety value. Anywhere real it turns "when any sensor activates" into "when any hardware in radio range activates". | | `pairing_window` | `2m` | The ceiling a `pairing open` call is clamped to. A request for an hour becomes two minutes rather than being refused. | | `base_topic` | `zigbee2mqtt` | Must match the daemon's own value. A mismatch is the most common cause of a node that connects to the broker happily and never sees a single device. | #### The Other Adapters Each one is independent: configure the ones you own, leave the rest out. An adapter with no configuration is never constructed. ``` # --- Z-Wave, through Z-Wave JS UI --- # Its MQTT gateway MUST be set to "ValueID topics". See the note below. [homeassist.zwave] broker = "tcp://127.0.0.1:1883" base_topic = "zwave" gateway_name = "zwave-js-ui" # --- Thread and Matter, through Matter Server --- # Thread devices additionally need a Thread Border Router. Matter over WiFi # needs only the server. [homeassist.matter] url = "ws://127.0.0.1:5580/ws" # --- Philips Hue bridge, on the LAN --- [homeassist.hue] # Use a DHCP reservation: bridges do not have stable hostnames. host = "192.168.1.50" # Leave empty on the first run, then pair against the bridge's link button. app_key = "" # --- Daikin air conditioning, on the LAN --- # Covers BOTH the legacy BRP069/BRP072 units and the newer Onecta firmware. # The dialect is detected on the first poll; there is no extra key for it. [homeassist.daikin] hosts = ["192.168.1.60"] # There is no event channel on this API, so this is the resolution of every # climate trigger. poll_interval = "30s" # --- UniFi Access and Protect, on the LAN --- # Two apps on one console, each with its own integration API and its own key. # Enabling one does not enable the other, and a key for one does not # authenticate to the other. Leave a key empty to disable that half. [homeassist.unifi] host = "192.168.1.1" access_key = "" # doors, NFC and card swipes, entry events, door stations protect_key = "" # cameras, doorbell cameras, motion events ``` ## Z-Wave JS UI must publish ValueID topics Its MQTT gateway offers two layouts and only one of them is machine-parseable. "Named topics" interpolates your own location and node names into the topic path, so `zwave/kitchen/ceiling/switch_binary/...` cannot be told apart from a node whose name contains a slash. ValueID topics are `zwave/ / / / `, which always parses. Set it in Z-Wave JS UI under Settings, Gateway, Gateway type. **The Hue app key is created against the link button.** There is no inclusion mode to open on a Hue bridge: it is already a coordinator with its own devices, and what the node needs is authorisation. Leave `app_key` empty, start the node, press the physical button on the bridge, and within thirty seconds call the pairing endpoint. The key is printed to the node log; paste it into `config.toml` so it survives a restart. It is deliberately not stored in the device registry, because a bridge credential does not belong somewhere a blueprint can read it. ``` curl -X POST https:///_system/homeassist/pair \ -H 'Authorization: Bearer ' \ -d '{"adapter":"hue"}' ``` #### UniFi Needs Two Keys, and Both Must Be Enabled Console-Side Access and Protect are separate apps on the same console, with separate integration APIs and separate keys. Enabling one does not enable the other, and a key issued by one does not authenticate to the other. This is the single most likely reason a UniFi adapter connects and reports nothing. | Key | Issued in | Enables | | --- | --- | --- | | `access_key` | Access, then Security, then Advanced, then API | Doors, card and NFC swipes, entry granted and denied, door stations, lock and unlock commands | | `protect_key` | Protect, then Settings, then Control Plane, then Integrations | Cameras, doorbell cameras, motion events | Give the console a DHCP reservation: consoles do not have stable hostnames. TLS verification is skipped for the same narrow reason as the Hue bridge, a self-signed certificate on a hard-coded LAN address with no chain to verify. **RUAL cannot appear inside the UniFi app.** UniFi has no third-party integrator surface that renders in its own UI; the integration APIs are read and command only. The node is a client of the console, not a plugin to it. #### Daikin Onecta Needs No Extra Configuration The `[homeassist.daikin]` section covers both firmware generations. The legacy BRP069 and BRP072 units expose a flat `k=v` API at `/aircon/*`; the newer BRP069C4x "Onecta" firmware replaced it with a nested JSON tree at `/dsiot/*`, which is a different protocol on the same port. A unit that answers 404 on the legacy endpoint is detected as Onecta on the first poll, and the node remembers which dialect each host speaks. Power on and off is reliable on Onecta. The temperature and mode decoding follows community reverse engineering and the hex encoding varies by model, so treat those readings as best-effort until a live unit confirms them. Every `/dsiot` reading is also carried untranslated on the device's `raw` map under `dsiot`, so a flow that finds a field mis-decoded can read the raw tree instead. #### History Recording Device analytics is **on** by default, because a home automation system that cannot answer "what has the temperature been doing" is missing the obvious question. It is bounded by a sampling policy rather than by luck; the reasoning is on [Device History and Energy](https://docs.rual.nl/home-automation/history-and-energy). ``` [homeassist.analytics] # Set false to record nothing. The history block then reports recording = false, # which is deliberately distinguishable from an empty result. enabled = true # How long readings are kept. Zero disables pruning entirely, which is only # correct where somebody else is watching the disk. retention = "2160h" # 90 days, the default flush_interval = "30s" ``` ## Step 5: First Boot Restart the node and read the log. Four lines tell you whether this worked: | Log line | Means | | --- | --- | | `HomeAssist: enabled for entity 1, adapters: [zigbee]` | The service started and constructed the adapters you configured. If an adapter you expected is missing, its config section is missing or empty. | | `zigbee2mqtt: bridge online` | The broker is reachable and the daemon is running. | | `zigbee2mqtt: N device(s) in mesh` | The device list arrived and was parsed. N is 0 on a fresh coordinator. | | `Restored N device(s) for entity 1` | Identity data came back from the store. Absent on a first boot, expected on every one after. | Two warnings are worth recognising immediately: - `[homeassist] enabled but no adapter is configured`. The service is running and no radio is attached to it, so the blocks stop reporting `HOMEASSIST_NOT_AVAILABLE` and start reporting nothing at all. This is loud rather than fatal on purpose: a node whose broker is briefly down must still boot. - `zigbee2mqtt: bridge OFFLINE`. The broker is reachable but Zigbee2MQTT is not running, or it has lost the coordinator. Different problem, different fix. Nothing here is fatal, and that is deliberate. A stick that is unplugged at boot, a broker that is down, a Hue bridge on a router that has not finished starting: all ordinary conditions in a house, none of them a reason for the node to refuse to serve auth, pages and blueprints. Adapters retry with backoff and report through the status API. ## Step 6: Prove the Radio Is Connected Two ways, and they answer slightly different questions. **From a blueprint**, the `get status` block reports one entry per adapter with its connection state, plus device counts. Its `healthy` pin is true only when every configured adapter is connected, and a node with no adapters at all is not healthy: the service being enabled with nothing wired is a misconfiguration, and reporting it green would hide exactly the case you are checking for. **Over HTTP**, the same payload: ``` curl -s https:///_system/homeassist/status \ -H 'Authorization: Bearer ' # {"entity":1,"adapters":[{"name":"zigbee","connected":true}], # "devices_total":0,"devices_adopted":0} ``` The home-automation routes require the `blueprints` / `simulating` scope, which is stronger than the scope that guards blueprint metadata. That is on purpose: a device list is a floor plan. It names every room, says which are occupied right now, and says which doors are open. A node with home automation switched off answers `503` with `HOMEASSIST_NOT_AVAILABLE` and a hint, rather than `404`. The route exists on every node, and "this build has no such endpoint" and "this node has home automation off" are exactly the two things a client discovering capabilities needs to tell apart. ## Next Steps - [Discovering and Adopting Devices](https://docs.rual.nl/home-automation/devices): Open pairing, adopt what appears, and give it a room. - [Troubleshooting](https://docs.rual.nl/home-automation/troubleshooting): No devices, bridge offline, triggers that do not fire. - [RUAL Core Nano](https://docs.rual.nl/architecture/core-nano): The machine this runs on, and the rest of its configuration. ## Frequently asked **What do I need to install to run home automation on a RUAL Nano?** Redis and an MQTT broker such as Mosquitto for the node itself, plus one daemon per protocol: Zigbee2MQTT for Zigbee, Z-Wave JS UI for Z-Wave, or Matter Server for Thread and Matter. RUAL does not drive the radios itself; it reads what those daemons publish and translates it. **How do I switch on home automation in a RUAL node?** Set enabled = true in the [homeassist] section of the node's config.toml and configure at least one adapter, for example [homeassist.zigbee] with a broker and a base_topic. It is off by default, and a node without it starts no radio goroutine and reports HOMEASSIST_NOT_AVAILABLE from every home-automation block. **Can Zigbee2MQTT run in Docker on a Mac?** Not with a USB coordinator. Docker Desktop on macOS runs containers inside a Linux VM with no USB passthrough, so the dongle cannot be handed to a container. Run Zigbee2MQTT natively under Node.js on a Mac; on Linux the ordinary Docker setup works with the coordinator passed through by its /dev/serial/by-id path. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Discovering and Adopting Devices · RUAL Documentation" description: "Pairing per radio, why adoption is a separate gate, names and rooms, forget versus unpair, and what survives a restart." canonical: https://docs.rual.nl/home-automation/devices language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Discovering and Adopting Devices A coordinator hears every device ever paired to its network. Adoption is the gate that turns a radio's earshot into a system you can reason about, and this is the loop around it. Devices appear on their own. You do not register them, import them or describe them: the adapter announces whatever the radio reports and the node records it. What you do decide is which of those devices are part of your system, and that decision has a name. ## Why Adoption Exists A discovered device is tracked, visible in the API, and has its state followed. It also **fires no triggers and refuses every command** until it is adopted. That is not ceremony. A Zigbee coordinator hears every device in radio range that has ever been paired to its network, including the ones somebody paired to test and forgot. A second-hand Z-Wave stick inherits whatever the previous owner left in its NVM. Without an explicit adopt, "when any sensor activates" means "when anything within thirty metres twitches", which is not an automation anybody can reason about. The gate is enforced in two places, the trigger dispatcher and the command path, rather than in each block. A new trigger block cannot forget it. There is exactly one exception, and it is the one that has to be: `device discovered` fires for unadopted devices, because noticing them is its entire purpose. | | Discovered | Adopted | | --- | --- | --- | | Appears in the device list | Only when you ask for unadopted devices | Yes, by default | | State is tracked | Yes | Yes | | Fires `device discovered` | Yes, once | Not applicable | | Fires any other trigger | No | Yes | | Accepts a command | No, `HOMEASSIST_DEVICE_NOT_ADOPTED` | Yes | | Records history | No | Yes, per the sampling policy | ## The Onboarding Loop - **Open the radio for joining.** The `pairing open` block, or `POST /_system/homeassist/pair`. The duration you ask for is clamped to the node's `pairing_window`, so a request for an hour becomes two minutes rather than being refused. Pass 0 to close pairing immediately. - **Put the device into pairing mode.** Usually a long press, sometimes a specific number of taps. This is the vendor's business, not RUAL's. - **It appears as discovered.** The `device discovered` trigger fires, once, the first time that device is ever seen. - **Adopt it, and give it a name and a room.** From that moment its triggers fire and blocks can command it. Step 4 can be automated: wire `device discovered` straight into `device adopt`, which takes optional name and room pins. That flow is worked through on [Example Flows](https://docs.rual.nl/home-automation/example-flows#auto-adopt). For a bench mesh, `auto_adopt = true` in the config skips the gate entirely, and it should stay off anywhere real. #### Adopting by Hand List what is waiting, then take it into service: ``` # Everything the radio has heard that you have not taken into service. curl -s 'https:///_system/homeassist/devices?unadopted=1' \ -H 'Authorization: Bearer ' # Adopt it, naming it and placing it in one call. curl -X POST 'https:///_system/homeassist/devices/zigbee:0x00158d0007e1a2b3/adopt' \ -H 'Authorization: Bearer ' \ -d '{"name":"Hallway motion","room":"Hallway"}' ``` `unadopted=1` is a separate query rather than the default, and the default list returns adopted devices only. A picker that offers devices a flow cannot command is a picker that produces flows which silently do nothing. Adoption is idempotent. Adopting an already-adopted device succeeds and reports `newly = false`, so a flow can call it without checking first. ## Pairing Is Different on Every Radio The `pairing open` block takes an adapter name and a duration, but what that means underneath varies more than the uniform interface suggests. | Adapter | What "open pairing" does | | --- | --- | | `zigbee` | Permit-join on the coordinator, for the requested seconds. The coordinator enforces the window itself. | | `zwave` | Starts inclusion, with S2 security where the device supports it. Z-Wave inclusion is not time-bounded by the controller the way permit-join is, so the node closes the window itself when the duration expires. | | `matter` | Opens the fabric's commissioning window so *another* controller can join. Adding a new device to Matter is not "open a window and wait": it needs the 11-digit pairing code or the `MT:` string printed on the device, which is a different operation. | | `hue` | Creates an application key. A Hue bridge has no inclusion mode: it is already a coordinator with its own devices, and what the node needs is authorisation. The bridge grants it only in the thirty seconds after somebody physically presses the link button, which is a better proof of presence than any credential we could store. | | `daikin` | Not supported, and it says so: `HOMEASSIST_PAIRING_UNSUPPORTED`. A Daikin unit is on the WiFi or it is not. Add its IP to `[homeassist.daikin] hosts`. | | `unifi` | Also unsupported, for a similar reason. UniFi devices are enrolled in the UniFi app, and the node reaches them through the console's integration APIs afterwards. What RUAL needs is a key per app, not an inclusion window: see [Setting Up a RUAL Nano](https://docs.rual.nl/home-automation/setting-up#config). | ## Identity, Names and Rooms Every device carries two identifiers, a name and a room: | Field | Example | What it is for | | --- | --- | --- | | `id` | `zigbee:0x00158d0007e1a2b3` | The platform identifier, ` : `. Stable across restarts and renames, which is why blueprints pin to it. The prefix means two radios can carry the same native address without colliding. | | `native_id` | `0x00158d0007e1a2b3`, `12`, a Hue UUID | The address in the adapter's own namespace. What you would type into Zigbee2MQTT or Z-Wave JS UI. | | `name` | `Hallway motion` | What a human calls it. Seeded from the vendor's friendly name, then yours. | | `room` | `Hallway` | Free text, set by you. Empty until somebody says. | Any block that addresses a device accepts all three of the platform id, the native id and the **exact** name, case-insensitively. Name resolution exists because typing "Hallway motion" into a pin is the common case and should work. It is deliberately last and deliberately exact: a prefix match would make renaming one device silently re-point another's flows. **Your name and room are yours.** Once set, a re-announce from the radio never overwrites them. A Zigbee re-pair resets the vendor's friendly name to the model number, and silently undoing "Hallway motion" would be worse than ignoring the vendor. Everything else merges: newly reported capabilities are added, a blank manufacturer or model is filled in. **Renaming in RUAL does not rename at the radio.** The Zigbee2MQTT friendly name and the Hue bridge name are unchanged, so those apps keep showing the old name. Two names for one device is mildly annoying; a rename that silently rewrites another system's topic structure is worse. #### Rooms Are Free Text There is no room registry to create rooms in. A room exists because at least one device is assigned to it, and `get rooms` returns the distinct non-empty room names across all adopted devices, sorted. Room filters compare case-insensitively, so "Hallway" and "hallway" are the same room, but "Hall" is a different one. Rooms are what make whole-house automations readable. A trigger with an empty device pin and `room = Hallway` is "any matching device in the hallway", which is one block instead of one flow per sensor. ## Where Device Knowledge Comes From The obvious way to build a device database is to scrape a public one. This does not, and the reason is that the best such database is already on the wire. Zigbee2MQTT publishes its full device definition, the `exposes` tree covering every attribute a device reports with its type, unit and range, on `zigbee2mqtt/bridge/devices`, refreshed whenever the mesh changes. That is the same data a scraped copy would be a stale version of, it covers roughly 4,000 devices, and it arrives already matched to the specific device on your specific mesh. Z-Wave JS does the same through its own config index. So the translation of that tree *is* the knowledge base, and a small profile table sits on top of it for the thing the wire cannot provide: devices whose exposes tree is technically correct but produces a bad default. Aqara sensors are the clearest case, reporting battery, voltage, link quality and device temperature alongside the one attribute anybody automates on. A profile guarantees the useful capabilities are present before the first report of that kind arrives, normalises the vendor string ("LUMI" becomes "Aqara"), and overrides the derived type where derivation is genuinely wrong, such as a button that would otherwise classify as a sensor. A device with no profile is not degraded. It gets exactly what its exposes tree said, which is almost always right. Profiles are polish, and they are applied on discovery *and* on adopt, so a device paired before its profile existed picks it up the next time somebody adopts it. ## Forget Is Not Unpair `device forget`, and `DELETE /_system/homeassist/devices/ `, drop the device from RUAL: its name, room and adoption are discarded. It does **not** unpair the device from the Zigbee, Z-Wave or Matter network. The device stays on the mesh and reappears, unadopted, on its next report. That is why forgetting is a plain DELETE with no confirmation dance. "Stop automating this" and "evict this from my mesh" are different intentions, and conflating them makes the safe one irreversible. Unpairing is done in Zigbee2MQTT or Z-Wave JS UI, where it belongs, and is deliberately not exposed as a block. A device that leaves the mesh on its own is treated the same way. When Zigbee2MQTT reports a device leaving, the node marks it unavailable and keeps it registered, because a device leaving is not you saying "forget this": it is unreachable, which is exactly what `available` reports. Deleting it there would silently discard its name, room and adoption the first time a battery died. ## What Survives a Restart Identity and state both survive, in two different places, for two different reasons. | | Identity | Last-known state | | --- | --- | --- | | What | Which devices exist, name, room, adoption and when, capabilities shown, manufacturer, model, matched profile | The last value per capability: whether the light is on, the last temperature, whether the door is open | | Where | The document store, alongside everything else on the node | Redis | | Why there | It is an operator's work and must not be lost | It is hot, small, rewritten constantly and worthless once stale: the profile of a cache, and the opposite of identity | | Written when | On the changes that matter: discovery, adoption, rename, a new capability | On every reading, coalesced per device and flushed every two seconds | | Expires | Never | After 30 days, refreshed on each write | Keeping state out of the document store is what stops a motion sensor in a busy hallway, reporting several times a minute, from becoming the node's dominant write load on the same backend that serves blueprints. Coalescing per device matters for the same reason: one Zigbee message routinely carries occupancy, illuminance, battery and link quality, which is four changed capabilities and one write. #### A Restored Value Is a Belief, Not an Observation Restoring state deliberately does **not** set `available` or touch `last_seen`. The node knows what the door was doing; it does not yet know whether the door is still there. A query that could not tell those apart would report a fleet of confidently-reachable devices seconds after boot, so `available` stays false until the device actually reports. Momentary capabilities are excluded from the restored map entirely. Replaying the last button press as current state would make it look freshly pressed. #### Why This Exists The naive version of this feature has a memorable failure mode. With an empty registry at boot, the first report from every adopted sensor has no previous value to compare against, and treating "no previous" as a change means a window that has been open all night announces itself as "just opened" at 03:00: the alarm flow runs, the lights come on. Two things prevent it. State is restored *before* the adapters connect, so the first real report has something to compare against and an unchanged re-announce stays unchanged. And independently of persistence, an unknown previous value seeds silently rather than firing, except for `smoke`, `gas` and `water_leak`, where a missed event is worse than a repeated one. The full rule is on [the trigger reference](https://docs.rual.nl/home-automation/triggers#unknown-previous). Forgetting a device deletes its stored state too, so re-adopting one later does not restore a belief from before it was removed. ## In RUAL Studio The device list, the per-device state and the history charts live under **Devices**, then **Home Automation**. That screen is what the `/_system/homeassist` routes exist for: the onboarding view (which uses the unadopted query), adopt, rename, forget, and a one-capability command so that having adopted a bulb you can check it actually responds without building a blueprint to find out. Every trigger and action block that addresses a single device carries the metadata for an **in-editor device picker**: the block declares that it wants a device selector, and which device type to offer, so `light on` offers lights and not thermostats. Offering a thermostat there would invite a flow that fails at runtime with `HOMEASSIST_CAPABILITY_UNSUPPORTED` for a mistake the editor could have prevented. ## The picker UI is not shipped yet The block metadata and the API it reads are in place; the studio side is not. Until it lands, set the device by typing an id or an exact name into the block's device pin, which is fully supported and is what the examples in this section do. The screen does not poll. Device events are pushed onto the WebSocket bus as they arrive, on the `homeassistdevices:updates: ` channel for one device and `homeassistdevices:updates:all` for the fleet, which is the subscription an overview wants. Polling is the obvious alternative and it is wrong here for a specific reason: the thing an operator is looking at while onboarding hardware is whether the sensor they just triggered lights up, and at a three-second poll that reads as "the sensor is broken" often enough to send somebody hunting a fault that does not exist. Unlike the trigger dispatcher, the broadcast does **not** filter on adoption, because the onboarding screen exists precisely to show devices that are not adopted yet. When the picker does land, one rule governs which wins: **the device pin beats the picker, always.** A flow that computes which lamp to address, from a room query or from the device pin of the trigger that started it, must not have that silently overridden by whatever somebody clicked in the editor months ago. The picker is a convenience for the fixed-device case; falling back the other way would make the wired case unreliable in a way that is invisible on the canvas, because both look like a configured block. ## The Device API All of it is under `/_system/homeassist` and all of it requires the `blueprints` / `simulating` scope. | Route | Does | | --- | --- | | `GET /status` | Per-adapter connection state and device counts. | | `GET /devices` | The device list. Filters: `type`, `adapter`, `room`, `capability`, plus `unadopted=1` or `include_unadopted=1`. | | `GET /devices/{id}` | One device, resolved by id, native id or exact name. | | `GET /devices/{id}/history` | Recorded readings, and buckets when `buckets` is set. | | `POST /devices/{id}/adopt` | Adopt, with an optional `name` and `room` body. | | `POST /devices/{id}/command` | Write one capability. One per call, deliberately: the caller here is a human poking a slider, not an automation. | | `POST /devices/{id}/rename` | Set name and/or room. | | `DELETE /devices/{id}` | Forget. Does not unpair. | | `POST /pair` | Open or close pairing on one adapter. | | `GET /rooms` | The distinct room names in use. | | `GET /capabilities` | The vocabulary itself: sensor capabilities, device types, per-capability units, decimals, axis bounds and chart type, and the adapter names. | `GET /capabilities` needs no running service and therefore no node with a radio, because the studio has to populate capability dropdowns while editing a blueprint that will run somewhere else entirely. It is also where the presentation contract lives: units, decimals and chart types come from the platform rather than from a second copy in the studio that can drift. ## Next Steps - [Trigger Reference](https://docs.rual.nl/home-automation/triggers): All 28 blocks that start a flow, and the three rules that decide how often they fire. - [Example Flows](https://docs.rual.nl/home-automation/example-flows): Including auto-adopting a newly discovered device into a room. - [Action Reference](https://docs.rual.nl/home-automation/actions): Adopt, rename, forget and pairing from a flow. ## Frequently asked **Why does my RUAL device not respond even though it appears in the list?** It is discovered but not adopted. An unadopted device is tracked and its state is followed, but it fires no triggers and refuses every command. Adopt it through the studio, through POST /_system/homeassist/devices/{id}/adopt, or from a flow with the device adopt block. **Why does RUAL make me adopt devices instead of using all of them?** A coordinator hears every device ever paired to its network, including ones somebody paired to test years ago, and a second-hand Z-Wave stick inherits whatever the previous owner left on it. Without an explicit adopt, a trigger like when any sensor activates would fire on hardware nobody remembers installing. **Does forgetting a device in RUAL unpair it from the radio?** No. Forget removes the device from RUAL, discarding its name, room and adoption, and the device stays paired to the Zigbee, Z-Wave or Matter network and reappears unadopted on its next report. Unpairing is done in Zigbee2MQTT or Z-Wave JS UI, because stop automating this and evict this from my mesh are different intentions. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Trigger Reference · RUAL Documentation" description: "All 28 start-of-flow device blocks (20 released, 8 coming soon), and the three rules that decide how often they fire: transitions, threshold crossings and unknown-previous seeding." canonical: https://docs.rual.nl/home-automation/triggers language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Trigger Reference The 28 blocks that start a flow when a device reports something, what fires each one, and the three rules that decide how often: transitions fire on the change, thresholds fire on the crossing, and an unknown previous value seeds instead of firing. A trigger block has a flow OUT pin and no flow IN pin, which is what makes it a start-of-flow block. It is not reached by the flow walk; the device event dispatcher seeds its out pins and starts the flow from there, the same way a schedule block works. Every trigger on this page fires only for **adopted** devices. The single exception is `device discovered`, whose purpose is to notice devices that are not adopted yet. ## The Rules That Surprise People #### Transition Triggers Fire on the Change, Not on Every Report A motion sensor does not send one message when somebody walks in. It re-announces `occupancy: true` every thirty seconds or so while a person is in the room, and a power meter reports every ten seconds whether or not anything changed. Every binary trigger requires the value to have *changed*. So `presence detected` in a hallway with somebody standing in it fires **once**, on entry. Without that rule a "turn on the lights" flow would re-run continuously and a "send me a notification" flow would be unusable. The one deliberate exception is `button pressed`. A button publishes `single` and stays `single` until the next press, so a value-equality test would swallow every repeat. A momentary capability has no meaningful "unchanged", so every press fires. #### Threshold Triggers Fire on the Crossing, Not on the Level `temperature changed` with `above = 25` fires **once**, when the room passes 25 °C. It does not fire again at 25.5, or 26, or every minute for the rest of the afternoon. It fires again only after the reading has gone back below 25 and crossed up again. This is the single most common way a threshold automation turns into a notification flood, so the rule is in the platform rather than in your flow. The exact test is: the new reading is above the threshold, and the previous one was at or below it. One deliberate wrinkle. A **first-ever** reading above the threshold counts as a crossing, because there is no previous value to have crossed from, and staying silent until the second reading would mean a node restart swallows an alarm. Setting neither `above` nor `below` turns the threshold off entirely and the trigger fires on every changed reading, which for a sensor reporting each minute is a busy flow. Setting both fires on either crossing. Note that 0 is a legitimate threshold and is treated as one: leaving a pin empty is what turns it off, not setting it to zero. #### An Unknown Previous Value Seeds Instead of Firing When the node has no previous value for a capability, an incoming reading is recorded **without** counting as a transition. Nothing fires; the value is simply learned. That rule exists because of what a restart otherwise does. With an empty registry, every adopted sensor re-announces the state it has been in for hours, and treating each first report as a change turns a routine restart into a burst of "door opened" and "light turned on" for things that did neither. A window open all night would announce itself as just opened at 03:00, run the alarm flow and switch the lights on. There are exactly three exceptions, where the asymmetry runs the other way: `smoke`, `gas` and `water_leak`. A duplicate "smoke detected" is an annoyance; a missed one is the thing the sensor exists to prevent. So a node that comes back up to an alarm already sounding says so, and accepts the occasional repeat. Occupancy, contact and vibration are deliberately not on that list: they are the high-frequency ones, they are what a restart most often finds mid-state, and nothing catastrophic follows from learning about them one report late. Momentary capabilities are also unaffected, because they have no persistent state to seed. Every button press is an event by definition. In practice you will rarely see this, because last-known state is restored from Redis before the adapters connect, so the first report after a restart usually *does* have something to compare against. The seeding rule is what makes the behaviour correct even when it does not: a fresh install, a flushed Redis, a device heard from for the first time. See [What Survives a Restart](https://docs.rual.nl/home-automation/devices#persistence). ## Every Trigger Has the Same Shape Two in-pins narrow which devices a trigger listens to, and both are optional. | In-pin | Effect | | --- | --- | | `device` | Only this device. Accepts the platform id, the native id or the exact device name. **Leave it empty and the trigger fires for every matching device**, which is what makes one block able to be the whole-house version. | | `room` | Only devices in this room, compared case-insensitively. Empty means every room. | Leaving the device pin optional is the design decision that keeps this usable. Requiring it would force one flow per sensor, which is the shape that makes home automation tedious. Out-pins are identical on all of them, and the two access-entry triggers add three more: | Out-pin | Type | Carries | | --- | --- | --- | | `flow` | flow | The chain to run. | | `device` | object | The whole device: id, name, room, type, capabilities, current state. Branch on model or battery without a second block. | | `device id` | value | The stable platform id, ready to feed another block's device pin. | | `name` | value | The friendly name. There because the common case is putting it in a message, and reaching into an object for that is friction. | | `room` | value | The device's room, empty if none. | | `type` | value | `sensor`, `light`, `switch`, `climate`, `cover`, `lock`, `button`, `door` or `camera`. | | `capability` | value | Which capability changed. The pin to branch on when using the catch-all trigger. | | `value` | value | The new value. | | `previous` | value | The value before this change. Empty on the first report. | | `at` | number | Unix seconds the device reported. | | `actor` | value | *Access granted and access denied only.* Who: the user or credential name, empty when the reader could not identify them. | | `method` | value | *Access triggers only.* How: `nfc`, `pin`, `wave`, `mobile`, `touchpass` or `manual`. | | `granted` | condition | *Access triggers only.* True when access was allowed. | ## Sensors, the General Ones | Block | Fires when | | --- | --- | | [`sensor activated`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_sensor_activated) | Any sensor capability transitions false to true: motion detected, a door opened, water found, smoke detected, gas detected, vibration. The extra `sensor type` pin narrows it to one of `occupancy`, `contact`, `water_leak`, `smoke`, `gas`, `vibration`; empty means any of them. | | [`sensor deactivated`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_sensor_deactivated) | The mirror: a sensor capability transitions true to false. | "Sensor" here means the binary capabilities that describe the world. It excludes `power`, which has its own light and switch triggers, and excludes `tamper` and `battery_low`, which are device health. Folding `power` in would make every whole-house sensor flow fire on its own lighting changes and livelock. On `sensor deactivated`, remember whose delay you are seeing. A motion sensor clears after *its own* configured stillness timeout, often 60 to 120 seconds, so this fires when the sensor says clear, not when the room empties. [sensor activated](https://docs.rual.nl/block-types/home/homeassist_trigger_sensor_activated) starts the flow when a sensor turns ON, and the two [value](https://docs.rual.nl/block-types/value/value_default) blocks are what narrow it: sensor type keeps it to leak detectors, room keeps it to the basement. Both are optional, and leaving them empty makes this one block the whole-house version. It fires on the TRANSITION, so a sensor sitting in a puddle and re-reporting fires the flow once, and the flow shuts the valve with [switch off](https://docs.rual.nl/block-types/home/homeassist_switch_off). [![Studio canvas example for the sensor activated block: one trigger for every sensor of one kind.](https://docs.rual.nl/canvas-examples/homeassist_trigger_sensor_activated.png)](https://docs.rual.nl/canvas-examples/homeassist_trigger_sensor_activated.png?v=3) ## Presence and Access | Block | Fires when | | --- | --- | | [`presence detected`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_presence_detected) | `occupancy` goes false to true. Equivalent to `sensor activated` with the sensor type set to occupancy, and it exists as its own block because it is the single most common automation there is. | | [`presence cleared`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_presence_cleared) | `occupancy` goes true to false. The delay before this fires is the sensor's, not the platform's. | | [`contact opened`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_contact_opened) | `contact` goes true, which means **open**. Zigbee and Matter both report contact true for a *closed* door on the wire; that inversion is corrected at the adapter, so `value` here is true when the door is open. | | [`contact closed`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_contact_closed) | `contact` goes false, meaning closed. | ## Alarms | Block | Fires when | | --- | --- | | [`water leak detected`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_water_leak) | `water_leak` transitions to true. On the transition, so a sensor sitting in a puddle and re-reporting fires once. | | [`smoke detected`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_smoke_detected) | `smoke` transitions to true. | ## These supplement a certified alarm, they do not replace one They depend on a mesh, a broker and a node all being up. Gas has no dedicated trigger; use `sensor activated` with the sensor type set to `gas`. ## Lights and Switches | Block | Fires when | | --- | --- | | [`light turned on`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_light_on) | A device classified as a **light** transitions `power` false to true. | | [`light turned off`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_light_off) | The same device type transitions `power` true to false. | | [`switch turned on`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_switch_on) | A device classified as a **switch** transitions `power` false to true. | | [`switch turned off`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_switch_off) | The same, true to false. | **These fire for changes RUAL did not make.** A physical wall switch, the Hue app, a scene fired by a dimmer: the radio reports the new state and the trigger fires. That is usually the point, and it is occasionally a trap, because a flow that turns a light on in response to a light turning on is a loop. The light/switch split follows the derived device type, not the vendor's opinion. A device with on/off but no brightness or colour is a switch, so metering plugs land here rather than under lights, even when the vendor announces them as lights. ## Buttons and Scene Controllers | Block | Fires when | | --- | --- | | [`button pressed`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_button_pressed) | Any momentary control emits an action: Aqara buttons, Hue dimmers, scene switches. Fires on **every** press, including a repeat of the same press type. | Branch on the `value` pin to tell single from double from hold. **The exact strings are the device's own.** Common ones are `single`, `double`, `hold` and `release`, but a four-button remote sends things like `1_single` and a Hue dimmer sends `initial_press` and `long_release`. Press the button once with the Zigbee2MQTT frontend open, or read the device state through `get device`, and use what it actually sends rather than what it ought to. ## Environment Readings | Block | Extra pins | Fires when | | --- | --- | --- | | [`temperature changed`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_temperature_changed) | `above`, `below`, in °C | The `temperature` capability changes. With a threshold set, only on the crossing. | | [`humidity changed`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_humidity_changed) | `above`, `below`, in % | The `humidity` capability changes. Same crossing semantics. | Other numeric capabilities have no dedicated trigger. For illuminance, CO₂, power draw or a blind's position, use `device state changed` and branch on the `capability` pin, or read the value with `get device state` inside a flow another trigger started. The threshold-crossing behaviour is specific to these two blocks. ## Device Health and the Fleet | Block | Fires when | | --- | --- | | [`battery low`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_battery_low) | The battery percentage crosses below the `below %` pin, default 20, **or** the device raises its own `battery_low` flag. Both dialects are handled because devices implement one or the other and rarely both. | | [`device unavailable`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_device_unavailable) | A device transitions to unreachable: a flat battery, out of range, a bulb switched off at the wall. It does **not** repeat while the device stays away. There is no matching "device came back" trigger; use `get status` or a periodic `get all devices` sweep if you need one. | | [`device discovered`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_device_discovered) | A new device appears on the radio, before it is adopted. Fires once per device, the first time it is ever seen. **The one trigger that fires for unadopted devices.** | | [`device adopted`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_device_adopted) | A device is taken into service, whether by the API, the studio or a flow calling `device adopt`. Use it to set a room, apply defaults or announce the new device. | ## The Catch-All | Block | Fires when | | --- | --- | | [`device state changed`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_trigger_device_state_changed) | Any capability of any adopted device changes. | **High volume by design.** A house with a dozen sensors produces hundreds of these an hour. Use it for logging, and for capabilities that have no dedicated trigger, then branch on the `capability` pin. Where a specific trigger exists, use the specific trigger: it is cheaper and it says what it means on the canvas. ## Doors, Access Control and Cameras These eight arrive with the UniFi Access and Protect adapter, and they are the automations people actually want from a door: when someone rings, when a card is swiped, when the gate opens, when an unknown card is refused at three in the morning. ## These eight are not yet released They are not in a released Core build yet, so unlike the blocks above they have no block pages here. Those appear once the blocks do. | Block | Extra pins | Fires when | | --- | --- | --- | | `doorbell rang` | | Somebody rings a doorbell or door station, from a UniFi door station or a Protect doorbell camera. Momentary: there is no "still ringing" state. | | `door or gate opened` | | `door_position` transitions to true, meaning open. This is the physical position, **not** the lock: a door can be unlocked and shut. | | `door or gate closed` | | `door_position` transitions to false. Pair it with a delay for "the gate has been open too long". | | `access granted` | `method` | Somebody is let in: a valid card, PIN, wave or mobile credential. The `method` pin narrows it to one of `nfc`, `pin`, `wave`, `mobile`, `touchpass`. Does not fire on a refusal. | | `access denied` | | Somebody is refused: an unknown card, a wrong PIN, a credential outside its schedule. | | `NFC card swiped` | `card` | A card or fob is presented, regardless of the decision that follows. The `card` pin narrows it to one token or actor. | | `camera motion detected` | | A UniFi Protect camera detects motion. One event per detection, not a sustained state. | | `security alarm activated` | | The controller's own alarm transitions to active. Distinct from the smoke, gas and leak triggers, which are individual detectors. | **The two entry triggers carry three extra out-pins**, because "someone came in" is only useful with who and how: `actor` (the user or credential name, empty when the reader could not identify them), `method`, and `granted`. They sit beside the standard pins rather than buried in the device object. **Swiped and granted are different questions.** `NFC card swiped` fires when the card is presented, before and regardless of the decision, which is what you want for "when my gym fob is tapped, turn on the porch light". `access granted` fires only when the door actually opened to somebody. A refused card fires the first and not the second. **One caveat on repeats.** Doorbell and camera motion events carry the moment they happened, so consecutive events differ and each one fires; two in the same second collapse into one. NFC carries the card token, so the *same* card presented twice in a row with nothing in between does not fire the second time. If you need every tap counted, read the access event instead, which carries the full payload. ## How Dispatch Actually Works Two properties are worth knowing because they explain both the speed and the limits. **Dispatch is in process, not queued.** Every other trigger in RUAL goes through the queue, which buys durability at the cost of a write and a poll interval. That trade is wrong here: the entire value of "when motion is detected, turn on the light" is that it happens before you have finished walking through the door, and a queue round trip is hundreds of milliseconds on a good day. The cost is that a node restart drops in-flight triggers, which is the correct behaviour anyway. Replaying a motion event from before a restart would turn a light on for somebody who left ten minutes ago. **The trigger index is cached, not searched per event.** A single occupancy sensor produces tens of events a minute, and searching your blueprints on each one would make a busy house the node's dominant read load. The index is built at startup, before the adapters connect, so the first motion event after a restart is not dropped into an empty index. It is rebuilt when you save a blueprint, over the same channel that invalidates the blueprint cache, so a studio edit takes effect without a restart. **Concurrent runs are capped at 16.** Beyond that, events are dropped with a warning in the log rather than queued: a backlog of stale "turn the light on" commands is worse than none. Sixteen is sized for a house, and hitting it usually means a flow is slow rather than that the house is busy. Only *active* trigger blocks in *saved* blueprints are indexed. ## Next Steps - [Action Reference](https://docs.rual.nl/home-automation/actions): The 24 blocks that command devices and query the registry. - [Example Flows](https://docs.rual.nl/home-automation/example-flows): These triggers wired into complete automations. - [Troubleshooting](https://docs.rual.nl/home-automation/troubleshooting): When a trigger does not fire, start here. ## Frequently asked **Does a RUAL motion trigger fire every time the sensor reports?** No, it fires on the change. A motion sensor re-announces occupancy every thirty seconds or so while somebody is in the room, and the trigger requires the value to have changed, so the flow runs once, on entry. Button presses are the deliberate exception, because a momentary pulse has no meaningful unchanged state. **How does a temperature threshold trigger work in RUAL?** It fires on the crossing, not the level. A trigger set to above 25 fires once when the room passes 25 degrees, not once per reading for the rest of the afternoon, and it fires again only after the reading has dropped back below 25 and crossed up again. A first-ever reading above the threshold counts as a crossing. **How do I write one RUAL automation for every sensor in a room?** Leave the trigger's device pin empty and set its room pin. The device pin is optional precisely so that one block can be the whole-room or whole-house version; requiring it would force one flow per sensor. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Action Reference · RUAL Documentation" description: "The 24 action blocks: lights, switches, climate, doors and gates, the generic capability write, the registry queries and fleet management." canonical: https://docs.rual.nl/home-automation/actions language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Action Reference The 24 blocks that command a device or ask the registry a question: lights, switches, climate, doors and gates, the generic capability write, the queries that cost nothing, and fleet management from a flow. An action block has a flow IN pin, which is what distinguishes it from a trigger: it is reached by the flow walk and sits in the middle of a chain like any other block. ## What Every Command Block Shares Blocks that address one device take a `device` pin: the platform id, the native address, or the exact device name. When the studio's device picker ships, a device selected there is used when the pin is empty, and **the pin always wins**. If neither is set the block reports `HOMEASSIST_DEVICE_REQUIRED` rather than guessing: "turn off the light" with no light named is not a whole-house command, it is a mistake. Command blocks report the same four out-pins: | Out-pin | Type | Carries | | --- | --- | --- | | `flow` | flow | Continues whether the command succeeded or not. | | `ok` | condition | True when the command reached the device. | | `device` | object | The device the command was sent to. | | `error` | value | Empty on success, otherwise a code plus the detail. | **Branch on `ok`, not on an error handler.** A command to an unreachable bulb is an ordinary event in a house, not an exceptional one, and forcing every lighting flow through error handling for a bulb somebody switched off at the wall would make them all unreadable. **Do not branch on the returned device state.** Hue and Daikin confirm synchronously, so their snapshot already reflects the command. Radio adapters do not: the Zigbee bulb has been told, and its own report arrives a moment later, so the snapshot you get back is the pre-command one. That is exactly why `ok` exists. #### Error Codes | Code | Means | | --- | --- | | `HOMEASSIST_NOT_AVAILABLE` | This node has `[homeassist] enabled = false`. Reported on the error pin rather than thrown, so a flow that runs on both a Nano and a cloud node can branch on it instead of blowing up. | | `HOMEASSIST_DEVICE_REQUIRED` | No device pin and no picked device. | | `HOMEASSIST_DEVICE_NOT_FOUND` | Nothing resolved from that id, native id or name. | | `HOMEASSIST_DEVICE_NOT_ADOPTED` | The device exists but is not in service. See [Discovering and Adopting Devices](https://docs.rual.nl/home-automation/devices). | | `HOMEASSIST_CAPABILITY_UNSUPPORTED` | The device never announced that capability, so nothing was sent to the radio. | | `HOMEASSIST_ADAPTER_REQUIRED` | `pairing open` with no adapter named. | | `HOMEASSIST_ADAPTER_NOT_FOUND` | That adapter is not configured on this node. | | `HOMEASSIST_COMMAND_REJECTED` | The device or its daemon refused the command. | | `HOMEASSIST_PAIRING_UNSUPPORTED` | That adapter has no inclusion mode. Daikin units join by WiFi; UniFi devices are enrolled in the UniFi app. | | `HOMEASSIST_NOTHING_TO_SET` | `climate set` with none of mode, temperature or fan wired. | | `HOMEASSIST_HISTORY_NOT_AVAILABLE` | Home automation is on but analytics is off. | The error string carries the code *and* the detail, for example `HOMEASSIST_DEVICE_NOT_FOUND: zigbee:0x1234`. The detail is the half you actually need when reading a flow trace. ## Lights | Block | Pins beyond `device` | Does | | --- | --- | --- | | [`light on`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_light_on) | `brightness %`, `kelvin`, `colour` | Turns the light on, applying everything set in one command. Unset pins leave the lamp's current value alone. | | [`light off`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_light_off) | | Turns it off. No side effects beyond the power state: brightness and colour are remembered by the lamp and restored when it is next turned on. | | [`light toggle`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_light_toggle) | | Reads the current power state from the registry and sends the opposite. | | [`light dim`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_light_dim) | `brightness %` | Sets brightness without changing anything else. | **One command, not three.** Sending power, then brightness, then colour as separate messages makes the bulb apply each as it arrives, so it visibly jumps to full white before settling. Batching them is why `light on` carries those pins instead of expecting three blocks in a row. **Kelvin, not mireds.** 2200 candle, 2700 warm white, 4000 neutral, 6500 daylight. The adapters convert. **Toggle decides rather than fails when the state is unknown.** Last-known state is restored at boot, so this is rarer than it used to be, but a lamp nobody has heard from since it was adopted still has no power state. A toggle that errors because the platform does not know is useless to the person standing in a dark room, so unknown means on. **`light dim` implies power.** Setting a brightness on a lamp that is off turns it on at that brightness rather than storing it for later, and setting 0 turns it off. Use `light off` when off is what you mean; it reads better on the canvas. [light on](https://docs.rual.nl/block-types/home/homeassist_light_on) carries brightness, colour temperature and colour on its own pins, and sends every value that is set in ONE command, so the lamp transitions once instead of visibly stepping through full white first. Brightness is a percentage and colour temperature is in KELVIN, not mireds: 2200 candle, 2700 warm white, 6500 daylight. The device pin takes a platform id, a native address or an exact device name. [![Studio canvas example for the light on block: on, warm and dimmed, in one command.](https://docs.rual.nl/canvas-examples/homeassist_light_on.png)](https://docs.rual.nl/canvas-examples/homeassist_light_on.png?v=3) ## Switches | Block | Does | | --- | --- | | [`switch on`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_switch_on) | Sets `power` true on a device classified as a switch. | | [`switch off`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_switch_off) | Sets `power` false. | Identical on the wire to `light on` and `light off`. They are separate blocks because pickers and flows read better when plugs are not offered as lights, and because a metering plug classifies as a switch even when its vendor announces it as a light. ## Climate | Block | Pins | Does | | --- | --- | --- | | [`climate set`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_climate_set) | `mode`, `target °C`, `fan` | Commands a Daikin air conditioner over its local API, or a Zigbee, Z-Wave or Matter thermostat over the radio. | - **Mode** is one of `off`, `auto`, `cool`, `heat`, `dry`, `fan`. Setting anything other than `off` implies switching the unit on: an author who says "heat" and gets a unit that stays off has been given a puzzle, not a feature. - **Fan** is `auto`, `silent`, `low`, `medium`, `high` or `turbo`. Devices with fewer speeds get the nearest they have. - **Empty pins keep the current value**, and wiring none of the three reports `HOMEASSIST_NOTHING_TO_SET` rather than claiming a command was sent. - Everything goes in one command for the same reason as lights. On Daikin that means a read-modify-write of the whole control block, because its API rejects partial updates and resets any field you omit. ## Doors and Gates These four command an access-controlled door through the UniFi adapter. ## These four are not yet released They are not in a released Core build yet, so they have no block pages here; those appear once the blocks do. | Block | Pins beyond `device` | Does | | --- | --- | --- | | `door unlock` | `stay unlocked` | Releases the lock. By default this is a **momentary** unlock: the controller holds the relay open for its configured time and then re-locks. | | `door lock` | | Locks the door. | | `gate open` | | Opens a gate or garage door. | | `gate close` | | Closes it. | **Momentary is the default because the alternative is a security footgun.** "Buzz someone in" is the common case, and a block that left the front door unlocked until something else remembered to lock it would eventually leave it unlocked overnight. Set `stay unlocked` when you genuinely mean until further notice, and pair it with a `door lock` somewhere. All four resolve to a single `lock_state` write, because that is what an access controller exposes: a relay. They exist as four blocks so the canvas and the picker read the way a person thinks. On a single-relay gate that toggles, `gate open` and `gate close` are the same pulse and the controller decides the direction from the gate's current position. `door_position` is read-only: it is a sensor, not a command. Writing it returns `HOMEASSIST_CAPABILITY_UNSUPPORTED` rather than failing silently. ## The Escape Hatch | Block | Pins | Does | | --- | --- | --- | | [`device set`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_device_set) | `capability`, `value` | Writes any capability by name. | This is how you reach a blind's `position`, a plain lock's `lock_state`, or anything else in the vocabulary that has no dedicated block. Values are coerced to what the device wants, and the capability name is normalised, so "motion" and "on_off" resolve rather than failing. What is *not* forgiven is a capability the device never announced: that returns `HOMEASSIST_CAPABILITY_UNSUPPORTED` without touching the radio. Silently sending an attribute a device does not implement is how you get a bulb that stops responding until it is power-cycled. ## Queries The query blocks read the in-memory device registry. **No radio traffic, no network call**, cheap enough to run inside a loop, which matters because the natural shape of "turn off every light" is a query feeding a map feeding a command. | Block | Filters | Returns | | --- | --- | --- | | [`get all devices`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_devices_all) | `type`, `room`, `adapter`, `include unadopted` | `devices`, `count`. Adopted only unless you ask otherwise. Filters combine with AND. | | [`get active sensors`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_sensors_active) | `sensor type`, `room` | Sensors currently reading true: motion present, doors open, leaks found. | | [`get lights that are on`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_lights_on) | `room` | Lights whose power state is known and true. | | [`get device`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_device_get) | `device` | `found`, `device`, `state` (value per capability), `available`. | | [`get device state`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_device_state) | `device`, `capability` | `state known`, `value`, plus `boolean` and `number` coercions for branching. | | [`get rooms`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_rooms) | | The distinct non-empty room names across adopted devices, sorted. | | [`get status`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_status) | | `adapters` (name and connected), `devices total`, `devices adopted`, `healthy`. | **Unknown state is excluded, never assumed.** A sensor the node has never heard from is left out of `get active sensors` rather than reported inactive, and a lamp whose power state nobody knows is left out of `get lights that are on`. `get device state` makes the same distinction explicit: when `state known` is false, `value` is empty and must not be read as false or zero. Since last-known state is restored at boot, these lists are populated immediately after a restart rather than empty. What is *not* restored is reachability: a restored value does not set `available`, so `get device` reports `available = false` until the device actually reports. If a flow needs "this device is definitely there right now", read `available` rather than inferring it from the presence of a value. `get device` reports `found = false` rather than erroring when nothing matches, so a flow can handle a missing device without an error branch. `get status` is how you notice a bridge that has gone away, which otherwise looks exactly like a quiet house. Its `healthy` pin is true only when every configured adapter is connected, and a node with no adapters at all is not healthy: enabled with nothing wired is a misconfiguration, and reporting it green would hide exactly the case somebody is checking for. [get lights that are on](https://docs.rual.nl/block-types/home/homeassist_lights_on) reads the in-memory device registry, so it makes no network call and is cheap enough to run in a loop. It returns only lights whose power state is KNOWN and true: a lamp the node has not heard from since it started is left out rather than assumed off. The count pin feeds [>](https://docs.rual.nl/block-types/condition/condition_gt) and [branch](https://docs.rual.nl/block-types/flow/branch), which is the shape of "if anything is still on, do something about it". [![Studio canvas example for the get lights that are on block: asking the registry which lights are on.](https://docs.rual.nl/canvas-examples/homeassist_lights_on.png)](https://docs.rual.nl/canvas-examples/homeassist_lights_on.png?v=3) ## Fleet Management | Block | Pins | Does | | --- | --- | --- | | [`device adopt`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_device_adopt) | `device`, `name`, `room` | Takes a discovered device into service. Reports `newly`, true only when this call is what adopted it. | | [`device rename`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_device_rename) | `device`, `name`, `room` | Sets the friendly name and/or room. Empty pins leave that field alone, so this can set a room without touching the name. | | [`device forget`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_device_forget) | `device` | Drops the device from RUAL. It stays paired to the radio and reappears, unadopted, on its next report. | | [`pairing open`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_pairing_open) | `adapter`, `seconds` | Puts a radio into inclusion mode. Pass 0 to close pairing. | **Adopt happens before rename**, not the other way round. Renaming a device that does not exist fails, and the flow this block is usually wired into (discovered, then adopt, then name it after the room) must not leave a half-onboarded device behind if the rename is what failed. If the adopt succeeds and the rename does not, `ok` stays true, `newly` is honest, and the error pin carries the rename failure. Adoption is the irreversible half and it worked. **The seconds pin on `pairing open` is a request.** It is clamped to the node's `pairing_window`, and the clamp is not an error: the useful outcome, pairing being open, happened either way. A flow that failed because it asked for an hour would be worse than one that got two minutes. **Forget does not unpair.** That is a separate, physically destructive operation, deliberately not exposed as a block. See [Forget Is Not Unpair](https://docs.rual.nl/home-automation/devices#forget). ## History | Block | Pins | Returns | | --- | --- | --- | | [`get device history`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_device_history) | `device`, `capability`, `from`, `to`, `buckets` | `readings`, `buckets`, `count`, `recording`, `unit`, `error`. | The only block here that reads the analytics store rather than the live registry, which is why it is the only one that can answer a question about the past. The window defaults to the last 24 hours, and `buckets` greater than 0 additionally returns aggregated windows. Watch `recording`: false means this node stores no history for that capability, which is a different answer from "nothing happened". The sampling policy, the bucket shapes and the energy dashboard are on [Device History and Energy](https://docs.rual.nl/home-automation/history-and-energy). ## Next Steps - [Example Flows](https://docs.rual.nl/home-automation/example-flows): These blocks wired into nine complete automations. - [Device History and Energy](https://docs.rual.nl/home-automation/history-and-energy): What gets stored, buckets versus raw, and the energy panel. - [Trigger Reference](https://docs.rual.nl/home-automation/triggers): The blocks that start these flows. ## Frequently asked **How do I tell whether a RUAL device command actually worked?** Branch on the ok out-pin. A command to an unreachable bulb is an ordinary event in a house, so failures are reported on ok and error rather than thrown. Do not compare the returned device state: radio adapters return the pre-command snapshot because the device's own report arrives a moment later. **How do I set brightness and colour temperature on a RUAL light?** Use the brightness, kelvin and colour pins on the light on block, which sends every value set on it in one command so the lamp transitions once instead of stepping through full white first. Colour temperature is in kelvin, not mireds: 2700 is warm white and 6500 is daylight. **Are the RUAL device query blocks expensive to call?** No. Get all devices, get active sensors, get lights that are on, get device and get rooms read the in-memory registry, so they make no network call and touch no radio, and they are cheap enough to call inside a loop. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Example Flows · RUAL Documentation" description: "Nine complete automations, block by block: motion lights after dark, a leak shutting a valve, a house that empties itself, an unknown card at 3am." canonical: https://docs.rual.nl/home-automation/example-flows language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Example Flows Nine complete automations, block by block: which trigger, how the chain is wired, why it is shaped that way, and the mistake each one is avoiding. Every flow here is built from the blocks on the [Trigger Reference](https://docs.rual.nl/home-automation/triggers) and [Action Reference](https://docs.rual.nl/home-automation/actions) pages, plus ordinary RUAL blocks you already use. Device references are written as an exact device name (`Hallway ceiling`) because that reads better on a page; a platform id such as `zigbee:0x00158d0007e1a2b3` works identically and is what the device picker will write once it ships. The pattern to notice across all nine: **the trigger does the filtering**. Its device and room pins are how you narrow the flow, not an `if` in the chain. A flow that fires for the whole house and then discards most events runs hundreds of times an hour for nothing. ## 1. Motion Lights, But Only After Dark **What it does.** Somebody walks into the hallway. If the hallway is actually dark, the light comes on, warm and dimmed. If it is the middle of the afternoon, nothing happens. | Block | Wired | | --- | --- | | `presence detected` | room `Hallway`, device pin empty | | `get device state` | device from the trigger's `device id`, capability `illuminance` | | `<` | the state block's `number` pin against `30` | | `branch` | flow from the state block, condition from the comparison | | `light on` | on the branch's true pin, device `Hallway ceiling`, brightness 40, kelvin 2700 | **Why it is shaped this way.** Three decisions are doing work here. The trigger's device pin is empty and its room pin is set, so this is one block for every motion sensor in the hallway rather than one flow per sensor. Add a second sensor to the room later and it works with no edit. The illuminance reading comes from the sensor that fired, by feeding the trigger's `device id` straight into `get device state`. Most motion sensors report lux as well as occupancy, and the reading from *that* sensor is the one that describes the space it is watching. Hard-coding a different sensor's id would work too, and is what you want when the motion sensor has no light meter. Brightness and kelvin ride along on `light on` rather than being set by a following `light dim`. One command means the lamp transitions once instead of visibly jumping to full white first. **The mistake it avoids.** Because `presence detected` fires on the transition, somebody standing still in the hallway does not re-run this every thirty seconds. Without that, the flow would fight anyone who dimmed the lamp by hand. **Turning it off again** is a separate flow, deliberately: `presence cleared`, room `Hallway`, into `light off`. Two small flows read better on the canvas than one with a branch, and they are independently editable. Remember that the delay before `presence cleared` fires is the sensor's own stillness timeout, usually 60 to 120 seconds, not something RUAL controls. If you want longer, put a `delay` block in front of the `light off`. The most common automation there is. [presence detected](https://docs.rual.nl/block-types/home/homeassist_trigger_presence_detected) fires on the false to true transition of occupancy in the hallway, so somebody standing still does not re-run it. Its device id feeds [get device state](https://docs.rual.nl/block-types/home/homeassist_device_state), which reads the illuminance the same sensor reports; [<](https://docs.rual.nl/block-types/condition/condition_lt) compares that against 30 lux and [branch](https://docs.rual.nl/block-types/flow/branch) only lets [light on](https://docs.rual.nl/block-types/home/homeassist_light_on) run when the room is actually dark. [![Studio canvas example for the presence detected block: motion, but only after dark.](https://docs.rual.nl/canvas-examples/homeassist_trigger_presence_detected.png)](https://docs.rual.nl/canvas-examples/homeassist_trigger_presence_detected.png?v=3) ## 2. A Door Opens While Nobody Is Home **What it does.** The front door opens. If no motion sensor anywhere in the house currently reads occupied, that is worth a notification. | Block | Wired | | --- | --- | | `contact opened` | device `Front door` | | `get active sensors` | sensor type `occupancy`, room empty | | `==` | the query's `count` against `0` | | `branch` | true pin into the notification of your choice | **Why it is shaped this way.** "Nobody home" is not a state RUAL tracks, and inventing one would be a worse answer than deriving it. `get active sensors` reads the in-memory registry, so asking the question costs nothing, and asking it at the moment the door opened is more accurate than a flag some other flow maintains. Use the trigger's `name` and `at` pins in the message. "Front door opened at 14:32" is a useful notification; "a contact sensor changed" is not. **The honest limit.** A sensor the node has not heard from at all is *excluded* from `get active sensors` rather than counted as inactive. Last-known state is restored at boot, so a restart no longer makes this flow believe the house is empty, but a device that has never reported since it was adopted is still invisible to it. Occupancy also decays: a sensor that read occupied when the node went down is restored as occupied, and stays that way until it reports again. **Occupancy is not presence.** A motion sensor reads false when a person sits still, so this fires if the door opens while somebody is reading on the sofa out of view. A phone or a presence-detection radar such as an FP2 (which reports `occupancy` continuously rather than on motion) is the better input if false positives are unacceptable. ## 3. A Leak Shuts the Valve and Raises an Alarm **What it does.** A leak sensor finds water. A smart valve closes, and somebody is told. | Block | Wired | | --- | --- | | `water leak detected` | device and room both empty: any leak sensor anywhere | | `switch off` | device `Water valve` | | `branch` | condition from the switch block's `ok` pin | | notification | on both branches, with different text | **Why it is shaped this way.** The device pin is empty on purpose. A leak is a leak, wherever it is, and enumerating sensors here means the one you install next year is not covered. Shutting the valve comes *before* the notification, because the valve is the part that limits the damage and the flow should not be waiting on an email API to do it. **The branch on `ok` is the important part**, and it is what separates this from a flow that looks fine and is useless. If the valve did not respond, the notification must say so. "Leak detected in the basement, valve closed" and "Leak detected in the basement, VALVE DID NOT RESPOND" call for different human behaviour, and a flow that sends the first message in both cases is worse than one that sends neither. **Repeated firing is not a problem here.** The trigger fires on the transition, so a sensor sitting in a puddle and re-reporting every thirty seconds does not send thirty notifications. It fires again only if the sensor dries out and gets wet again, which is information. **Say this out loud when you build it:** this supplements a plumber and a water alarm, it does not replace them. It depends on a mesh, a broker and a node all being up. ## 4. A Double Press Sets a Scene **What it does.** One button, three behaviours: single press toggles the lamp, double press sets a warm evening scene across several lamps, hold turns everything off. | Block | Wired | | --- | --- | | `button pressed` | device `Living room button` | | `switch` | expression from the trigger's `value` pin | | `light toggle` | on the `single` case | | three `light on` blocks | on the `double` case, each with its own brightness and kelvin | | `light off` blocks | on the `hold` case | **Why it is shaped this way.** A scene in RUAL is not an object, it is a few blocks in a row. That is more typing than picking a saved scene in a vendor app, and it buys something: the scene can read the time, check who is home, or vary by season, because it is a flow rather than a stored list of values. **Find out what your button actually sends before you write the cases.** The strings on the `value` pin are the device's own. Aqara buttons send `single`, `double` and `hold`; a Hue dimmer sends `initial_press`, `short_release` and `long_release`; a four-button remote prefixes each with the button number. Press it once with the Zigbee2MQTT frontend open, or read `get device` and look at `state.action`, and use what it sends rather than what it ought to. **Why every press fires.** Button events are momentary pulses: the device publishes `double` and stays `double` until the next press. Every other trigger in this section requires the value to have changed, which would swallow a second identical press. `button pressed` is the deliberate exception, so pressing the same button twice fires the flow twice. ## 5. A Temperature Crossing Runs the Air Conditioning **What it does.** The office passes 25 °C during the day, so the air conditioner starts cooling to 22 on a quiet fan. When it drops back below 22, the unit goes off. | Block | Wired | | --- | --- | | `temperature changed` | room `Office`, `above` 25 | | `climate set` | device `Office AC`, mode `cool`, target 22, fan `silent` | | **and a second, separate flow:** | | | `temperature changed` | room `Office`, `below` 22 | | `climate set` | device `Office AC`, mode `off` | **Why it is shaped this way.** This is the flow that most obviously depends on crossing semantics. A thermometer reporting every minute on a hot afternoon produces hundreds of readings above 25, and a trigger that fired on the *level* would send hundreds of identical commands to the air conditioner. Because `above` fires on the crossing, the unit is told once. **The two thresholds are deliberately different numbers.** Cooling starts at 25 and stops at 22, so the unit is not switching on and off every time the reading wobbles across a single value. That gap is hysteresis, and picking it is the actual design work in this flow; the platform will not invent it for you. Mode, target and fan are set in one `climate set` rather than three. On a Daikin unit that matters more than it looks: its local API rejects partial updates and resets anything you leave out, so the adapter reads the whole control block, applies your changes and writes it back as one operation. Three separate blocks would be three read-modify-write cycles, and an occupant can hear the unit stepping through them. Setting a mode other than `off` implies switching the unit on, so there is no separate power step. **The honest limit.** The Daikin adapter polls: with the default 30 second interval, the temperature RUAL reacts to can be up to 30 seconds old, and "the AC turned on" fires up to an interval late. For an air conditioner that is fine. A Zigbee or Z-Wave thermostat pushes and has no such delay. ## 6. Energy Over a Threshold Raises a Warning **What it does.** A metering plug reports power draw. When something on that circuit pulls more than 2400 W for more than a minute, warn. | Block | Wired | | --- | --- | | `device state changed` | device `Utility plug` | | `==` | the trigger's `capability` pin against `power_watts` | | `>` | the trigger's `value` pin against `2400` | | `AND` | both conditions | | `branch`, `delay`, `get device state`, `>`, `branch` | wait 60 seconds, re-read `power_watts`, and only warn if it is still high | **Why it is shaped this way.** There is no `power changed` trigger. Only temperature and humidity have dedicated threshold triggers, so anything else numeric goes through `device state changed` with a check on the `capability` pin. The catch-all is high volume across the whole house, which is exactly why the device pin is set here: narrowed to one plug, it fires only for that plug's reports. **The delay-then-recheck is the point of the flow.** A kettle pulls 2400 W for ninety seconds every morning and that is not news. What you want to know about is a load that is *still* drawing when you look again. Doing that with a second read rather than with a stateful counter keeps the flow readable and stateless: the registry already knows the current value, so re-asking is free. **Watch the concurrency budget.** Device triggers run at most 16 flows at once, and beyond that events are dropped with a log line rather than queued. A `delay` inside a device flow holds one of those 16 slots for its whole duration. One minute is fine. If you find yourself wanting ten, move the waiting into the queue or a repeating event, and keep the device flow short. **For totals rather than spikes**, do not accumulate in a flow at all. The node already records energy at a fixed interval and can hand you consumption per hour or per day: see [Device History and Energy](https://docs.rual.nl/home-automation/history-and-energy). ## 7. The House Empties, Every Light Goes Off **What it does.** The last occupancy sensor clears. Every light that is currently on is switched off, without naming any of them. | Block | Wired | | --- | --- | | `presence cleared` | device and room empty | | `get active sensors` | sensor type `occupancy` | | `==` and `branch` | count against 0: nobody anywhere | | `get lights that are on` | room empty | | `select field values` | the `devices` array, field `id` | | `map` | that array of ids | | `execute` and `light off` | the map's function pin drives the execute block, whose `value` pin is one device id, into the `light off` device pin | **Why it is shaped this way.** This is the flow the query blocks exist for. `get lights that are on` reads the in-memory registry with no network call, so iterating it costs nothing, and the result is derived from the live state rather than from a list of light names you maintain by hand. Install a lamp next year and it is covered. `select field values` sits between the query and the map because the query returns device *objects* and `light off` wants a device reference. Pulling the `id` field out first gives the map an array of plain ids, which is exactly what the execute block's `value` pin then carries into each command. **Only lights that are actually on are touched**, which is the difference between this and looping over every light. Lamps whose state the node does not know are excluded rather than assumed off, so the flow never blasts commands at the whole house on a guess. **The re-check on the trigger matters.** `presence cleared` with an empty device pin fires when *any* sensor clears, which happens constantly in an occupied house as people move between rooms. Asking `get active sensors` whether anything is still occupied, and stopping if it is, is what turns "a sensor cleared" into "the house is empty". For a per-room version, set the room pin on both the trigger and the query, and drop the whole-house check. [get lights that are on](https://docs.rual.nl/block-types/home/homeassist_lights_on) reads the in-memory device registry, so it makes no network call and is cheap enough to run in a loop. It returns only lights whose power state is KNOWN and true: a lamp the node has not heard from since it started is left out rather than assumed off. The count pin feeds [>](https://docs.rual.nl/block-types/condition/condition_gt) and [branch](https://docs.rual.nl/block-types/flow/branch), which is the shape of "if anything is still on, do something about it". [![Studio canvas example for the get lights that are on block: asking the registry which lights are on.](https://docs.rual.nl/canvas-examples/homeassist_lights_on.png)](https://docs.rual.nl/canvas-examples/homeassist_lights_on.png?v=3) ## 8. Auto-Adopting a New Device Into a Room **What it does.** While you are commissioning a floor, every device that pairs is adopted, named after its model and placed in the room you are working in, without a trip to the API for each one. | Block | Wired | | --- | --- | | `device discovered` | device and room empty | | `get value variable` | a variable holding the room you are commissioning, for example `commissioning_room` | | `device adopt` | device from the trigger's `device id`, room from that variable, name from the trigger's `name` pin | | `branch` | on the adopt block's `newly` pin, into a notification or a log document | **Why it is shaped this way.** `device discovered` is the only trigger that fires for unadopted devices, which is what makes this possible at all. It fires once per device, the first time it is ever seen. `device adopt` takes name and room pins so the whole onboarding is one block rather than an adopt followed by a rename. Internally it does adopt first and rename second, and if the rename fails the adoption still stands and is reported honestly: `ok` stays true, and the error pin carries the rename failure. That ordering exists precisely so a failed rename does not leave a half-onboarded device behind. **Branch on `newly`, not on `ok`.** Adoption is idempotent, so calling it on an already-adopted device succeeds with `newly = false`. Notifying on `ok` would announce devices that were already yours. **Prefer this to `auto_adopt = true`.** The config flag adopts everything the radio can hear, forever, including whatever a neighbour paired to a mesh that overlaps yours. This flow is a switch you control: keep it deactivated, activate it while you are physically commissioning, and deactivate it when you are done. The trigger index rebuilds when you save, so activating and deactivating takes effect without a restart. **A safer variant:** instead of adopting, notify. Wire `device discovered` into a notification carrying the trigger's `name` and `device id`, and adopt from the studio when you recognise it. That is the right default for a mesh that is not brand new, because the devices a coordinator hears are not all yours. ## 9. An Unknown Card at Three in the Morning **What it does.** Somebody is refused at the front door. During the day that is a colleague with an expired badge and not worth a message; at night it is worth waking up for. Meanwhile every successful entry is logged with who and how. ## This flow uses blocks that are not yet released The `access denied` and `access granted` triggers arrive with the UniFi adapter, which is not in a released Core build yet. Everything else on the canvas is ordinary blocks. | Block | Wired | | --- | --- | | `access denied` | device empty: any door | | `date now`, `branch` | outside working hours | | notification | with the trigger's `name`, `actor` and `method` pins | | **and a second flow, for the log:** | | | `access granted` | device empty | | `create document` | a storage document carrying `actor`, `method`, `name` and `at` | **Why it is shaped this way.** The `actor`, `method` and `granted` pins are on the trigger itself rather than inside the device object, because "someone came in" is only useful with who and how. A notification that says "Access denied at Front door, unknown card, nfc" is actionable; "a door reported something" is not. **Denied and swiped are different triggers.** `access denied` fires on the decision. If you want to react to a specific card regardless of what the door decided, for example turning on the porch light when a particular fob is tapped, use `NFC card swiped` with its `card` pin, which fires when the card is presented and before the decision. **Write the log with a flow, not with history.** Access events are recorded in device history as events, so they are not lost, but "who came in last month" is a table you will want to query, filter and export. A storage document per entry gives you that; the analytics store is shaped for charts. **The gate variant:** `doorbell rang` into `gate open` is one wire and a genuinely bad idea. Put a branch in between that checks something: a delivery window, a person at home, a specific NFC token. A doorbell is an unauthenticated button anyone can press. ## Patterns Worth Reusing | When you want | Reach for | | --- | --- | | One flow covering a whole room or the whole house | Leave the trigger's device pin empty, set the room pin | | A condition about the world right now | A query block: they read memory and cost nothing | | To know whether a command actually landed | Branch on `ok`, not on the returned device state | | To act on a device the trigger identified | Feed the trigger's `device id` into the next block's device pin | | To act on many devices at once | Query, `select field values` for `id`, `map`, command | | A numeric capability with no dedicated trigger | `device state changed` narrowed to one device, then check the `capability` pin | | To avoid reacting to a momentary spike | `delay`, then re-read with `get device state` | | To answer a question about the past | `get device history`, not a variable you maintain | ## Next Steps - [Device History and Energy](https://docs.rual.nl/home-automation/history-and-energy): Questions about the past: what is recorded, and how to read it back. - [Troubleshooting](https://docs.rual.nl/home-automation/troubleshooting): When one of these does not fire, this is the checklist. - [Common Blueprint Patterns](https://docs.rual.nl/blueprints/common-patterns): The general-purpose shapes these flows are built from. ## Frequently asked **How do I turn on a light only when it is dark in RUAL?** Wire presence detected into get device state reading the illuminance capability of the same sensor, compare that number against a lux threshold, and put a branch in front of light on. Most motion sensors report lux as well as occupancy, and the reading from the sensor that fired describes the space it is watching. **How do I turn off every light in RUAL without naming them?** Feed get lights that are on into select field values for the id field, then into map, and drive light off from the execute block inside the map. The query returns only lights whose power state is known and true, so lamps the node has not heard from are left alone rather than blasted with commands. **How do I automatically adopt new smart home devices in RUAL?** Wire the device discovered trigger into the device adopt block, which takes optional name and room pins so onboarding is a single block. Keep that flow deactivated and activate it only while you are physically commissioning hardware, which is safer than the auto_adopt config flag because that adopts everything the radio can hear, forever. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Device History and Energy · RUAL Documentation" description: "Why the node records what changed rather than what was said, buckets versus raw readings, meter deltas and the energy view." canonical: https://docs.rual.nl/home-automation/history-and-energy language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Device History and Energy The node records what changed rather than what was said, at a resolution chosen per capability. That policy is what makes device history a feature instead of a disk-filling support ticket, and it decides how you read the data back. A dozen sensors in a normal house produce tens of reports a minute. A motion sensor re-announces while somebody is in the room, a power meter reports every ten seconds, and every Zigbee message carries a link-quality stamp. Storing one row per report is a few million rows a month whose information content is nearly zero. So nothing here stores what a device *said*. It stores **what changed**, at a resolution chosen per capability. Measured against a twelve-device house: roughly 40 reports a minute in, roughly 25 rows an hour out. ## Every Capability Has a Kind, and the Kind Decides Everything The organising idea is that a value's *behaviour over time* determines its storage policy, its chart and its aggregation, and all three can be derived from one attribute instead of special-cased in every consumer. | Kind | Examples | Stored | Charted as | Aggregates to | | --- | --- | --- | --- | --- | | **alarm** | occupancy, contact, water leak, smoke, gas, vibration, tamper, battery low, door position, security alarm | On transitions only. A door open for eight hours is two rows, not four hundred and eighty. | Timeline | Fraction of the window it was true | | **measure** | temperature, humidity, pressure, illuminance, CO₂, VOC, PM2.5, power draw | On change beyond a deadband, plus a heartbeat. | Line | min, max, average, last | | **meter** | energy in kWh | At a fixed interval, not on change. | Bars | Delta: consumption in that window | | **state** | power, brightness, colour temperature, position, HVAC mode, fan mode, lock state, target temperature | On change. | Steps | min, max, average, last | | **event** | button presses, doorbell rings, access entries, NFC swipes, camera motion | Every occurrence, because every occurrence is the data. | Not charted | Count | | **diagnostic** | battery percentage, link quality | Sparsely: a wide deadband and a long heartbeat. | Line, not by default | min, max, average, last | Two of those rows are the interesting ones. **A meter is not a measurement.** Energy in kWh is a monotonic lifetime counter, so charting the raw value is a meaningless upward ramp and "on change" means "on every report". It is sampled at a fixed five minutes instead, because evenly spaced points are what make bucket deltas comparable, and the useful view is consumption per hour or per day rather than the ever-rising total. **A setpoint is not a measurement either.** `target_temperature` is a state somebody chose: it steps rather than drifts, so it is charted as steps and stored on change. ## Deadbands and Heartbeats The **deadband** is the smallest change worth recording, in the capability's own unit. It is the single most important number in this design: a temperature sensor reporting 21.03, 21.04, 21.03 every thirty seconds produces 2,880 rows a day that say nothing, and a deadband of 0.1 collapses that to the handful of readings where the room actually changed. The **heartbeat** is the longest gap allowed between stored points even when nothing changed. Without it a deadband makes a stable sensor indistinguishable from a dead one, because both produce no rows. With it, a flat line is visibly flat and a gap is visibly a gap. | Capability | Deadband | Heartbeat | Why that number | | --- | --- | --- | --- | | `temperature` | 0.1 °C | 15 min | Below the accuracy of the sensors this runs against. | | `humidity` | 1 % | 15 min | Charted 0 to 100 regardless of the data range, because "is this normal" is the question being asked. | | `pressure` | 0.5 hPa | 30 min | Weather moves slowly. | | `illuminance` | 5 lx | 15 min | Lux spans five orders of magnitude between a dark room and direct sun, so a fixed deadband is wrong at one end. 5 keeps the dark end useful, which is the end automations care about. | | `co2` | 10 ppm | 15 min | | | `voc` | 5 ppb | 15 min | | | `pm25` | 1 µg/m³ | 15 min | | | `power_watts` | 5 W | 5 min | Watts, not percent: the interesting events are appliances switching on and off. A kettle is 2000 W and a standby LED is 0.3 W, and 5 W separates "something happened" from meter jitter. | | `energy_kwh` | fixed 5 min interval | | See above: change is not a signal on a counter that always rises. | | `brightness` | 1 % | | State, stored on change. | | `color_temp` | 50 K | | Below what anyone sees. | | `battery` | 1 % | 24 h | A battery moves a few percent a week. "When did this start dropping" is a real question; a daily heartbeat answers it. | | `link_quality` | 20 | 6 h | Recorded because "this sensor stopped working" is nearly always a mesh problem and the shape of the decline is the evidence. Wide deadband: LQI is noisy by tens between consecutive reports. | Alarms, states with no deadband, enum states and button events have neither: they store on every change, and events store on every occurrence. **The comparison is against the last *stored* value, not the last seen one.** Comparing against the last seen value would let a slow drift through one increment at a time and never record anything. ## What Is Recorded at All - **Only adopted devices.** A coordinator hears every device ever paired to its network; recording all of it would fill the store with the history of hardware nobody uses. - **Only capabilities that opt in.** A capability has to be listed in the metadata table with recording switched on. Anything absent records nothing, so adding a capability to the vocabulary cannot silently start filling a disk. `color_hex` is the one capability in the vocabulary that is deliberately not recorded. - **Booleans are stored as 0 and 1**, not as text, so one numeric column serves both timeline and line charts, and "how long was this true" is an ordinary integral rather than a string parse. Enum states such as `hvac_mode` are stored as text. - **The access log is recorded but not charted.** Doorbell rings, entry decisions, NFC swipes and camera motion are events, so every occurrence is stored. Who came in, when and how is exactly the history somebody wants, and it is a table rather than a curve. See [the access flow](https://docs.rual.nl/home-automation/example-flows#access) for writing a queryable log alongside it. This store is separate from the last-known-state cache in Redis that survives a restart. The two answer different questions: the cache holds one current value per capability so a restart does not re-fire every active trigger, and this store holds the trail of how that value got there. See [What Survives a Restart](https://docs.rual.nl/home-automation/devices#persistence). ## How It Is Written The recorder never writes on the ingest path. It appends to an in-memory buffer, and a background flush writes the batch every 30 seconds by default, or immediately when the buffer reaches 500 readings, because a burst is exactly when the data is interesting. Batching matters far more than the interval: 25 rows in one write beats 25 writes. A clean shutdown flushes what is buffered, so a restart does not leave a half-minute hole in every chart that spans it. A *failed* write is dropped with a warning rather than retried: analytics is the least important thing this node does, and a retry queue that grows while the store is unhealthy is how a nice-to-have takes down the automation that actually matters. Reads fold the buffer back in, so a chart opened seconds after a sensor fired shows that reading rather than waiting up to a flush interval. That matters because the single most common use of a history screen is looking at something that just happened. ## Reading It Back [`get device history`](https://docs.rual.nl/block-types/globals%2Chome%20automation/homeassist_device_history) is the only block that reads the analytics store rather than the live registry. | Pin | Meaning | | --- | --- | | `device`, `capability` | What to read. The device resolves by id, native id or exact name, same as every other block. | | `from`, `to` | The window. Empty means the last 24 hours, which is not arbitrary: the store records changes rather than samples, so a day of a quiet sensor is a handful of rows and a day is the span somebody means by "what has this been doing". Wiring the two the other way round swaps them rather than erroring. | | `buckets` | 0 returns raw readings only. Greater than 0 additionally returns that many aggregated windows. | | Out-pin | Carries | | --- | --- | | `readings` | The stored readings, oldest first: `device_id`, `capability`, `value`, `text`, `at`. | | `buckets` | Aggregated windows: `at`, `min`, `max`, `avg`, `last`, `delta`, `count`. | | `count` | How many raw readings came back. | | `recording` | False when this node stores no history for that capability. | | `unit` | The display unit, for example `°C` or `kWh`. | **Check `recording` before you interpret an empty result.** Empty with `recording = true` means nothing happened in that window. Empty with `recording = false` means this node was never going to tell you: either analytics is switched off, or that capability is not one that is stored. Those are different answers and a dashboard that conflates them lies. [get device history](https://docs.rual.nl/block-types/home/homeassist_device_history) reads the analytics store, which records what CHANGED rather than every report. With from and to left empty the window is the last 24 hours, and buckets set to 24 additionally returns one aggregated window per hour on the buckets pin. Watch the recording pin: it is false when this node stores no history for that capability, which is a different answer from "nothing happened". [![Studio canvas example for the get device history block: a day of temperature, bucketed for a chart.](https://docs.rual.nl/canvas-examples/homeassist_device_history.png)](https://docs.rual.nl/canvas-examples/homeassist_device_history.png?v=3) ## Buckets Versus Raw Readings Raw readings are the record: exactly what was stored, when. Buckets are for drawing. Ask for both when you are charting, because they answer different questions and re-deriving one from the other in the browser means shipping the aggregation rules to a second place where they can drift. The bucket shape follows the capability's kind, which is the entire reason kinds exist: | Kind | The pin that matters | Why | | --- | --- | --- | | meter | `delta` | Consumption in that window: this bucket's last value minus the previous bucket's last value. The raw lifetime total charts as a meaningless ramp. | | alarm | `avg` | The fraction of the window the value was true, which renders as a duty-cycle band. "The hallway was occupied 40% of that hour." | | everything else | `min`, `max`, `avg`, `last` | The ordinary summary of a gauge. | `delta` is only emitted for meters. Emitting a zero on every other kind would invite a chart that plots it. **Empty windows are omitted, not zero-filled.** A gap in a temperature chart means "the sensor did not report", and drawing it as 0 °C invents a cold snap. If your chart library wants a dense series, fill the gaps on the client where you can choose how to render them, rather than having the platform invent readings. **A meter that goes backwards contributes nothing.** A negative delta means the counter reset: a device replaced, or firmware reflashed. Reporting a large negative consumption would poison every total above it, so that one bucket reports no delta rather than a wrong one. #### Reading It Over HTTP ``` curl -s 'https:///_system/homeassist/devices/zigbee:0x00158d0007e1a2b3/history?capability=temperature&buckets=24' \ -H 'Authorization: Bearer ' ``` The response carries the readings, the buckets, and a `meta` object with the capability's kind, label, unit, decimals and chart type. That last part is the presentation contract: units, decimals, axis bounds and chart type come from the platform, so a chart cannot drift from what the platform actually records. It also carries `truncated`. The window defaults to the last 24 hours and the reading count is capped; a chart that silently truncated would be worse than one that shows fewer points, so the response says which happened. `from` and `to` are unix seconds, `buckets=0` asks for raw only. ## Energy Energy is the capability the meter kind was built for, and the shape of the answer is different from every other reading in this section. A metering plug reports two things: `power_watts`, the instantaneous draw, and `energy_kwh`, the lifetime counter. They answer different questions, and using the wrong one is the usual mistake. | Question | Read | | --- | --- | | What is drawing power right now | `power_watts`, live, through `get device state` | | Did the load spike this afternoon | `power_watts` history, bucketed: `max` per bucket | | How much did this use yesterday | `energy_kwh`, bucketed: sum the `delta` pins | | How much has it used since it was installed | `energy_kwh`, the last raw reading | For a per-day view, ask for the window you want with one bucket per day and read `delta`. For a month, 30 buckets over 30 days. Because the meter is sampled at a fixed five minutes, buckets of any width are directly comparable, which would not be true if it were sampled on change. In RUAL Studio this is the **Devices**, then **Home Automation** screen: per-capability history over 1h, 24h, 7d and 30d ranges, plus an energy panel per device. It reads exactly the routes above, and it renders units and chart types from the capability metadata rather than from its own copy. **The in-editor device picker on blocks is a separate piece of that work and is not shipped yet.** ## Retention Readings are kept for 90 days by default and pruned hourly. Hourly rather than daily, because a node that is only up for a few hours a day would otherwise never prune at all. ``` [homeassist.analytics] # Records nothing when false. The history block then reports recording = false, # which a flow can tell apart from an empty result. enabled = true # Zero disables pruning entirely: only correct where somebody else is watching # the disk. retention = "2160h" flush_interval = "30s" ``` ## What This Is Not - **It is not a metering-grade record.** It is sampled by design, so it answers trends and totals well and forensic questions poorly. "Roughly how much did the boiler use last week" is a question it answers accurately. "What exactly was the reading at 14:32:07" is not. - **If you need every reading**, for billing or for compliance, write them yourself: a `device state changed` trigger into `create document` gives you an unsampled record in a storage you control, with the cost and the volume that implies. Choose that deliberately, not by accident. - **Unadopted devices have no history at all**, including for the period before you adopted them. Adoption is when recording starts. - **History does not survive a store you do not keep.** On a Nano this is the same SQLite file as everything else, so the backup that covers your documents covers this too. Test the restore. ## Next Steps - [Example Flows](https://docs.rual.nl/home-automation/example-flows): Including a power-threshold warning that re-reads before it warns. - [Storages](https://docs.rual.nl/blueprints/storage): For when you decide you want the unsampled record after all. - [Troubleshooting](https://docs.rual.nl/home-automation/troubleshooting): Empty charts, and how to tell which kind of empty they are. ## Frequently asked **Does RUAL store every reading a smart home sensor sends?** No, it stores what changed. Alarms and states are recorded on transitions, measurements when they move beyond a per-capability deadband plus a heartbeat so a stable sensor stays distinguishable from a dead one, meters at a fixed five-minute interval, and button presses individually. A twelve-device house produces roughly 40 reports a minute and about 25 stored rows an hour. **How do I get energy consumption per day from RUAL?** Read the energy_kwh capability with get device history and ask for one bucket per day, then use each bucket's delta pin. Energy is a lifetime counter, so the raw value is an ever-rising ramp; the delta is the consumption in that window. A bucket where the counter went backwards, meaning the meter was reset, reports no delta rather than a wrong negative one. **Why is my RUAL device history chart empty?** Check the recording pin first. False means this node stores no history for that capability, because analytics is switched off or the capability is not recorded. True with no readings means nothing was stored in that window. Unadopted devices record nothing at all, including for the time before you adopted them. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Troubleshooting Home Automation · RUAL Documentation" description: "Symptom first: no devices, a trigger that will not fire, a door sensor that reads backwards, an empty chart." canonical: https://docs.rual.nl/home-automation/troubleshooting language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Troubleshooting Home Automation Symptom first: no devices, a trigger that will not fire, a door sensor that reads backwards, an empty chart. Most of these look identical from the outside, so each entry starts with the check that tells them apart. A home-automation stack is four processes, and three of them are not RUAL. That is why the first move is almost always to find out *which* layer is quiet, rather than to change a setting and see. ## The Three Checks That Localise Almost Everything Run these in order. Each one clears a layer. ``` mosquitto_sub -h 127.0.0.1 -t '#' -v # Expect zigbee2mqtt/bridge/state -> {"state":"online"} # and zigbee2mqtt/bridge/devices -> a JSON array ``` ``` curl -s https:///_system/homeassist/status \ -H 'Authorization: Bearer ' ``` ``` curl -s 'https:///_system/homeassist/devices?include_unadopted=1' \ -H 'Authorization: Bearer ' ``` | Where it stops | The problem is | | --- | --- | | Check 1 shows nothing | The radio daemon, or the broker. Not RUAL. | | Check 1 works, check 2 says `connected: false` | The node's broker settings, or credentials. | | Check 2 is fine, check 3 is empty | Almost always `base_topic`. | | Check 3 lists the device with `"adopted": false` | Adoption. This is the most common answer on the whole page. | ## No Devices Appear In order of how often it is each one: - **`base_topic` does not match.** The node subscribes under the base topic in `[homeassist.zigbee]`; Zigbee2MQTT publishes under `mqtt.base_topic` in its own configuration. Both default to `zigbee2mqtt`, so a mismatch only happens when somebody changed one. The failure is silent and total: the node connects to the broker happily and sees nothing. Compare the two files; do not trust memory. - **The adapter was never constructed.** An adapter with no configuration section, or with an empty `broker`, is not created at all. Check the boot line: `HomeAssist: enabled for entity 1, adapters: [zigbee]` lists exactly what exists. If it says `adapters: []`, you also had the warning `[homeassist] enabled but no adapter is configured`. - **Home automation is off.** Blocks report `HOMEASSIST_NOT_AVAILABLE` and the API answers 503 with a hint. Set `[homeassist] enabled = true`. - **The coordinator has no devices.** A fresh stick genuinely has an empty mesh, and `zigbee2mqtt/bridge/devices` is an empty array. Pair something. - **Broker credentials.** If the broker requires a username and password, both the node and each daemon need it. One side unconfigured looks exactly like a broken radio. Note that the Zigbee coordinator itself is deliberately excluded from the device list: it is not something an automation addresses, and letting it through would put a permanently unadoptable "Coordinator" entry at the top of every picker. Z-Wave node 1 is skipped for the same reason, it is the controller stick. ## Bridge Offline Is Not the Same as Broker Unreachable Both present as "no devices", and the fix is completely different, so the node logs them separately. | Log line | Means | Fix | | --- | --- | --- | | `zigbee2mqtt: bridge OFFLINE` | The broker is reachable and RUAL is connected to it. Zigbee2MQTT is not running, or it is running and has lost the coordinator. | Restart Zigbee2MQTT. If it starts and immediately reports offline again, the dongle is the problem: check the serial port, the adapter setting and the cable. | | MQTT connect failures with backoff | The broker itself is unreachable, or refusing the connection. | Check Mosquitto is running, check the host and port, check credentials. | | `connected: true` in status, no device traffic | Both are fine and the topic is wrong. | `base_topic`. | **The adapter status pin tracks the broker connection, not the bridge.** `get status` reporting `connected: true` means RUAL has an MQTT session, which is a weaker statement than "the radio works". That is why check 1 exists. #### Two Nodes, One Broker If a second RUAL node shares the broker, give each one its own `client_id`. Two MQTT connections with the same client identifier kick each other off, forever, and the resulting symptom is a device list that appears and disappears every few seconds on both nodes. ## A Trigger Does Not Fire **The device is not adopted.** That is the answer most of the time. An unadopted device is tracked and visible, its state is followed, and it fires nothing. The gate is enforced centrally, so no trigger is exempt except `device discovered`. After that, in order: - **The blueprint is not saved, or the block is not active.** The trigger index only contains active trigger blocks from saved blueprints. It rebuilds on save, so no restart is needed, but an unsaved edit does not exist as far as the dispatcher is concerned. - **The device pin does not resolve.** It accepts the platform id, the native id or the **exact** name, case-insensitively. A near-miss on the name silently matches nothing. Paste the id from `get all devices` when in doubt. - **The room does not match.** Room comparison is case-insensitive but not fuzzy: "Hall" and "Hallway" are different rooms, and a device with no room set matches no room filter. - **The device is the wrong type.** `light turned on` fires only for devices classified as lights and `switch turned on` only for switches. A metering plug is a switch even if the vendor calls it a light. Check the `type` field in `get device`. - **The value did not change.** Transition triggers require a change. If a sensor is already true and re-reports true, nothing fires, by design. - **The threshold was not crossed.** "Above 25" fires when the reading passes 25 from at or below it. If the room has been at 27 all day, it fired once this morning and will not fire again until it drops below 25 and comes back up. - **The capability is not the one you think.** `sensor activated` with a sensor type set fires only for that capability. Read the device's actual capability list with `get device`; a sensor that reports `occupancy` will not fire a trigger narrowed to `contact`. - **The concurrency cap.** At most 16 device flows run at once, and beyond that events are dropped with a warning naming the trigger type and device. If you see that line, something in your flows is slow: a `delay` inside a device flow holds a slot for its whole duration. A quick way to isolate steps 3 to 7: temporarily put a `device state changed` trigger with no device and no room in front of a debug block. If that fires and your specific trigger does not, the problem is a filter, not the plumbing. ## A Contact Sensor Reads Backwards **It should not, and if it does, look at your flow before you look at the sensor.** Zigbee's contact attribute is true when the magnet is present, which is when the door is *closed*. Matter's BooleanState is the same way round. The platform inverts both at the adapter, so in RUAL **`contact = true` means OPEN**, everywhere, on every protocol. This is the single most common source of backwards door automation in every home automation system there is, which is exactly why it is corrected once, at the boundary, rather than left to each flow. So if a door flow behaves backwards, the usual causes are: - The flow uses `contact closed` where it meant `contact opened`, on the assumption that it had to compensate. - The flow reads `contact` through `get device state` and inverts it by hand, double-correcting. - The magnet is mounted the wrong way round, or on the frame rather than the door, so the sensor genuinely reports the opposite of what the door is doing. Open the door and watch the value with `get device` or in Zigbee2MQTT: the sensor is the authority on its own wiring. Colour temperature has the same shape of correction: RUAL speaks **kelvin**, the wire speaks mireds. If a lamp goes cold when you asked for warm, check you passed 2700 and not 370. ## Z-Wave Devices Never Appear **The gateway type must be "ValueID topics".** Z-Wave JS UI can publish under two topic layouts and only one of them is machine-parseable. "Named topics" interpolates your own location and node names into the path, so `zwave/kitchen/ceiling/switch_binary/...` cannot be told apart from a node whose name contains a slash. ValueID topics are `zwave/ / / / `, which always parses. The node cannot work with the other layout and does not try. Set it in Z-Wave JS UI under **Settings**, **Gateway**, **Gateway type**. Then check the topics directly: ``` mosquitto_sub -h 127.0.0.1 -t 'zwave/#' -v # Right: zwave/12/49/0/Air_temperature {"time":...,"value":21.5} # Wrong: zwave/kitchen/ceiling/49/0/Air_temperature {...} ``` Also check `gateway_name`. It is the `ZWAVE_GATEWAY- ` suffix on the API topics and must match the MQTT name configured in Z-Wave JS UI, default `zwave-js-ui`. With it wrong, values still arrive and get parsed, but the node inventory request goes nowhere: devices appear with names like "Z-Wave node 12" instead of the names you gave them, and with fewer capabilities than they should have. ## A Daikin Unit Reports Nothing, or Reports Odd Temperatures Both firmware generations are supported and the node picks the dialect itself: the legacy BRP069 and BRP072 units speak a flat `k=v` API at `/aircon/*`, and the newer BRP069C4x "Onecta" firmware speaks a nested JSON tree at `/dsiot/*`. A unit that answers 404 on the legacy endpoint is detected as Onecta on the first poll, and the node remembers which dialect each host speaks. So a Daikin unit that reports nothing at all is usually a network problem rather than a firmware one: wrong IP, the unit on a different VLAN, or a DHCP lease that moved. Confirm with `curl http:// /common/basic_info`. **Odd temperatures on an Onecta unit are a known limitation, not a fault.** Power on and off is reliable; the temperature and mode decoding follows community reverse engineering, the hex encoding varies by model, and it has not been verified against a live unit. If a reading looks wrong, read the untranslated tree instead: the full `/dsiot` response is on the device's `raw` map under `dsiot`, reachable through `get device`. A report of which model produced which wrong value is the thing that fixes this properly. Other Daikin specifics worth knowing: - **It is polled, not pushed.** The default 30 second `poll_interval` is the resolution of every climate trigger on that unit, and the state you read can be up to one interval old. - **Hosts are configured, not discovered.** Daikin adapters do not advertise reliably over mDNS. Put the IP in `[homeassist.daikin] hosts`, and give it a DHCP reservation so it does not move. - **Pairing does not apply.** `pairing open` on the Daikin adapter returns `HOMEASSIST_PAIRING_UNSUPPORTED`. The unit is on the WiFi or it is not. - **A unit that is off reports mode `off`**, not its last mode. The unit itself remembers "heat" all summer; reporting that verbatim would make `hvac_mode == "heat"` true for a switched-off air conditioner. ## UniFi Connects but Reports Nothing **Almost always the keys.** Access and Protect are separate apps on the same console, with separate integration APIs and separate keys. Enabling one does not enable the other, and a key issued by one does not authenticate to the other. - **No doors, NFC or entry events:** the Access half. Enable the API in Access, then Security, then Advanced, then API, and put that key in `access_key`. - **No cameras, doorbells or motion:** the Protect half. Enable it in Protect, then Settings, then Control Plane, then Integrations, and put *that* key in `protect_key`. - **Neither:** check the host. Consoles do not have stable hostnames, so use the IP with a DHCP reservation. **A field decodes to the wrong thing.** Possible, and expected to be found this way: the adapter follows Ubiquiti's documented APIs but its field decoding has not been confirmed end to end against a live console. Every payload is also carried untranslated on the device's `raw` map, so read that through `get device` and use the raw value while the mapping is corrected. A report naming the console version and the raw payload is what fixes it. **RUAL does not appear inside the UniFi app, and cannot.** UniFi has no third-party integrator surface that renders in its own UI. The integration APIs are read and command only, so the node is a client of the console rather than a plugin to it. **An NFC trigger fires once and then stops for the same card.** The NFC value is the card token, so presenting the same card twice in a row with nothing in between is not a change and does not re-fire. Use `access granted` or `access denied` when you need every attempt, since those carry the full event. ## Hue and Matter Specifics | Symptom | Cause and fix | | --- | --- | | Hue adapter starts and does nothing, with a warning about no app key | The bridge has not authorised this node. Press the link button and call `POST /_system/homeassist/pair` with `{"adapter":"hue"}` within thirty seconds. The key is printed to the log; paste it into `[homeassist.hue] app_key` so it survives a restart. It is deliberately not persisted automatically, because a bridge credential does not belong somewhere a blueprint can read it. | | A Hue motion sensor appears once, not four times | Correct. One Hue device carries motion, temperature, light level and battery as separate services; surfacing those as four devices would put four entries in the picker for something the household calls "the hall sensor". | | Hue bulbs are missing after moving them to your own coordinator | A bulb is paired to the bridge or to your stick, never both. Re-paired to your coordinator it arrives through the Zigbee adapter with a new device id, and the old Hue-adapter entry stops reporting. | | A two-gang Matter switch appears as one device | A known limitation. Matter endpoints are collapsed onto one device, whose `power` is whichever endpoint reported last. Modelling endpoints properly needs a sub-device concept the platform does not have yet. | | Matter pairing does not find a new device | Matter commissioning is not "open a window and wait". It needs the 11-digit setup code or the `MT:` string printed on the device. `pairing open` on the Matter adapter opens the fabric so another controller can join, which is a different operation. | ## A Command Does Nothing | Error pin | What to do | | --- | --- | | `HOMEASSIST_DEVICE_NOT_ADOPTED` | Adopt it. Commands to unadopted devices are refused by design. | | `HOMEASSIST_DEVICE_NOT_FOUND` | The reference did not resolve. The error carries what you passed; compare it against `get all devices`. | | `HOMEASSIST_CAPABILITY_UNSUPPORTED` | The device never announced that capability, and nothing was sent to the radio. Check the device's capability list; capabilities accumulate, so a device may reveal one only after it first changes. | | `HOMEASSIST_DEVICE_REQUIRED` | No device pin and no picked device. | | `ok = false`, empty error | The adapter refused or the send failed. Check the node log for the adapter's own message. | | `ok = true`, nothing happens | The command reached the daemon. Watch the broker: a `zigbee2mqtt/ /set` publish that produces no state message back means the device did not act, which is a mesh or device problem rather than a RUAL one. | **Do not diagnose by comparing the returned device state.** Radio adapters return the pre-command snapshot: the bulb has been told, and its own report arrives a moment later. Only Hue and Daikin confirm synchronously. `ok` is the pin that answers "did this land". ## What a Restart Does and Does Not Restore Last-known device state is persisted to Redis and restored before the adapters connect, so `get lights that are on` and `get active sensors` answer sensibly immediately after a restart rather than returning nothing. What is **not** restored is reachability. A restored value is a belief about what the device was doing, not evidence that it is still there, so `available` stays false and `last_seen` is untouched until the device actually reports. A device list read immediately after a restart therefore looks like a house of unreachable hardware with plausible values, which is exactly what it is. It fills in as devices report, within a polling interval for mains-powered devices and considerably longer for sleepy battery ones. | Symptom after a restart | Cause | | --- | --- | | Values look right, everything reports `available: false` | Normal. Nothing has reported yet. | | All state is empty, not just availability | Redis was flushed, this is a fresh install, or the entries expired. State is kept for 30 days, refreshed on each write. | | A sensor reads occupied or open when it is not | A restored belief that has not been corrected yet. It stands until the device next reports, which for a battery sensor that only speaks on change can be a while. | | Every trigger fired at once on boot | Should not happen: an unknown previous value seeds silently, and restored state gives the first report something to compare against. If you see it, check that Redis is reachable from the node, because without it the seeding rule is the only protection and a first-ever active reading of smoke, gas or water leak still fires by design. | A flow that runs at boot and depends on knowing a device is really there should read `available` rather than inferring it from the presence of a value. ## A History Chart Is Empty **Read the `recording` pin first.** It separates two very different situations that look identical: - `recording = false`: this node stores no history for that capability. Either `[homeassist.analytics] enabled = false`, or the capability is one that is not recorded at all (`color_hex` is the notable one). - `recording = true` with no readings: nothing was stored in that window. For an alarm capability that genuinely means the door never moved. For a measurement it can also mean the value never moved beyond its deadband, in which case you should still see heartbeat rows. Then check the obvious ones: the device was not adopted during that window (unadopted devices record nothing, including retrospectively), the window is longer than the 90 day retention, or the capability name is not the canonical one. **Gaps in a chart are real.** Empty buckets are omitted rather than zero-filled, because drawing a missing temperature reading as 0 °C invents a cold snap. A gap means the device did not report. **A single missing bar on an energy chart** is usually a meter reset: a device replaced or firmware reflashed makes the counter go backwards, and that bucket reports no delta rather than a large negative one that would poison every total above it. ## Devices Drop Out Intermittently This is nearly always physical, and nearly always the same two causes. - **The coordinator is plugged straight into the machine.** USB 3 ports and NVMe drives emit broadband noise across 2.4 GHz. Use the extension cable that came with the dongle, half a metre, away from the case. On a hub the same rule applies: dongles on their cables, not packed into adjacent ports. - **The mesh has no routers where it needs them.** Battery devices do not route. A mains-powered plug or bulb in the middle of a long run does more for reliability than anything in software. `link_quality` is recorded for exactly this: "this sensor stopped working" is nearly always a mesh problem, and the shape of the decline is the evidence. Chart it over 7 days for the device that drops, with `get device history` and capability `link_quality`. A device that leaves the mesh is marked unavailable and kept registered, with its name, room and adoption intact. It is not forgotten, because a dead battery is not you saying "remove this". ## A Button Trigger Fires but the Branch Never Matches The strings on the `value` pin are the device's own, not a RUAL vocabulary. Aqara sends `single`, `double`, `hold`; a Hue dimmer sends `initial_press`, `short_release`, `long_release`; multi-button remotes prefix the button number. Press the button once and read `state.action` through `get device`, or watch the broker, and branch on what it actually sends. Note that Zigbee2MQTT clears the action by publishing an empty string immediately after; that empty value is dropped rather than reported, so it does not double every press. ## Next Steps - [Setting Up a RUAL Nano](https://docs.rual.nl/home-automation/setting-up): The runbook, including the verification steps these checks refer back to. - [Discovering and Adopting Devices](https://docs.rual.nl/home-automation/devices): Adoption, which is the answer to most trigger problems. - [Debugging Blueprints](https://docs.rual.nl/troubleshooting/debugging): The console, simulation and error pins, for the flow half of the problem. ## Frequently asked **Why does RUAL show no home automation devices at all?** Most often the base_topic in the node's config does not match the one Zigbee2MQTT publishes under, which fails silently and completely. Check in order: does mosquitto_sub show traffic on the broker, does GET /_system/homeassist/status report the adapter connected, and does the device list include unadopted devices. **What is the difference between a Zigbee bridge being offline and the broker being unreachable in RUAL?** A bridge offline message means the broker is reachable and RUAL is connected to it, but Zigbee2MQTT is not running or has lost the coordinator. Broker connect failures mean the broker itself is unreachable. Both present as no devices, so the node logs them separately. **Why do my Z-Wave devices not appear in RUAL?** Z-Wave JS UI must have its MQTT gateway type set to ValueID topics. The named-topics layout interpolates your own location and node names into the topic path, which cannot be parsed back into a node id, so RUAL cannot use it. ValueID topics look like zwave/12/49/0/Air_temperature. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Examples · RUAL Documentation" description: "Reference applications. Each one shows the document design first, then the flows that read and write it." canonical: https://docs.rual.nl/examples language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Examples Reference applications. Each one shows the document design first, then the flows that read and write it. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Example: Simple Blog · RUAL Documentation" description: "Posts, slugs, tags and a published state." canonical: https://docs.rual.nl/examples/simple-blog language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Example: Simple Blog A complete working blog: post model, list and detail pages, publishing flow, and the APIs behind them. A blog is the smallest complete app: one storage, two pages, one publishing flow. This example shows every piece working together: clone the structure for any content-driven site. ## The Post Document ``` { "_meta": { "cms": 1782080295147, "created": 1782080295, "entity": 1, "expiry": -1, "guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "removed": 0, "ums": 1782166695813, "update_hash": "60e27209fa93063b5605e41605c2722ed428cae0", "updated": 1782166695 }, "author_guid": "a23575f49d0af385314c1f02280163374297e018a692e5e4ab85eb307ebf6ebc", "author_name": "sami b", "body": "# Why we rebuilt\n\nEverything starts with...", "published_at": 1782166695, "slug": "why-we-rebuilt-the-docs", "status": "published", "tags": [ "docs", "meta" ], "title": "why we rebuilt the docs" } ``` One `posts` storage, everything embedded: the author is denormalized (`author_name` stored next to `author_guid`) so list pages never join. The core storage principle from [Storages](https://docs.rual.nl/blueprints/storage). ## Public Pages - **Index page** (`/blog`, scope `*public`): a `function_search` for `status = published`, sorted by `published_at` descending, limit 20. The list-query shape from [templates](https://docs.rual.nl/block-types/block-templates#api-endpoint). Iterate results into post cards. - **Post page** (`/blog/post`, `*public`): reads `slug` from the page params, fetches with [`function_search_single_result`](https://docs.rual.nl/block-types/storage/function_search_single_result) (term on `slug` + term on `status = published`: drafts 404 for the public), renders title/body with state elements. - **Feed**: the same list query behind an API block → JSON for external readers. See [Creating a REST API](https://docs.rual.nl/tutorials/rest-api). ## The Publishing Flow - An editor form (`*loggedin` + custom `editor` scope) writes drafts with [`function_create_document_from_mutations`](https://docs.rual.nl/block-types/storage/function_create_document_from_mutations). The form template from [Block Templates](https://docs.rual.nl/block-types/block-templates#form-handling). - The slug is generated once from the title (lowercase, dashes) with a uniqueness check before create. - Publishing is a status mutation (`draft → published` + `published_at` timestamp) via [`function_update_document_mutations`](https://docs.rual.nl/block-types/storage/function_update_document_mutations). - Edits create revisions automatically; the list only ever shows `published`. ## Search Add a search box feeding [`query_bool_simple_query_string_field`](https://docs.rual.nl/block-types/query/query_bool_simple_query_string_field) on `title` + `body` into the same list query: lowercase the fields at write time, per [search optimization](https://docs.rual.nl/blueprints/storage-examples#search-optimization). ## Takeaways to Reuse - One storage per content type; embed what lists display. - Status fields drive visibility. Never delete content, transition it. - Public reads and editor writes are separate pages with separate scopes. - The same query serves the page, the API, and the search box. ## Next Steps - [First CRUD App](https://docs.rual.nl/getting-started/first-crud-app): The tutorial this example builds on. - [Storage Examples](https://docs.rual.nl/blueprints/storage-examples): More production-shaped data models. - [E-Commerce Catalog](https://docs.rual.nl/examples/ecommerce-catalog): The same pattern at product scale. ## Frequently asked **How do I keep draft blog posts out of the public list in RUAL?** Drive visibility with a status field. The public pages query status = published, and the post page adds a term on status so drafts return 404 for the public. Publishing is a mutation from draft to published plus a published_at timestamp: never delete content, transition it. **Why does the RUAL blog example store the author name on every post?** RUAL storage has no joins, so the author is denormalized: author_name is stored next to author_guid, and list pages render names without fetching a second document. **How do I add a search box to a RUAL blog?** Feed query_bool_simple_query_string_field on title and body into the same list query that serves the index page, and store those fields in lowercase at write time so the case-sensitive search matches. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Example: E-Commerce Catalog · RUAL Documentation" description: "Products with variants, stock and category filtering." canonical: https://docs.rual.nl/examples/ecommerce-catalog language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Example: E-Commerce Catalog Products, variants, categories, stock, and storefront search. A complete catalog with the data model that makes it fast. A catalog looks relational but shouldn't be modeled that way on RUAL: products carry everything their page shows. This example covers the model, the storefront, and stock handling that can't double-sell. ## The Product Document ``` { "_meta": { "cms": 1780531200118, "created": 1780531200, "entity": 1, "expiry": -1, "guid": "c45797061f2c15a75361042244385e96419d230ab804d7fa08cd52981fd78192", "removed": 0, "ums": 1782166695204, "update_hash": "f1a9c2d47b6035e8419dc0a7532be96481cd30fa", "updated": 1782166695 }, "category": "apparel", "image": "/static/products/tshirt-black.webp", "name": "essential cotton t-shirt", "price": 2490, "sku": "TSHIRT-BLK-M", "status": "active", "variants": [ { "size": "m", "sku": "TSHIRT-BLK-M", "stock": 14 }, { "size": "l", "sku": "TSHIRT-BLK-L", "stock": 3 } ] } ``` - Variants live inside the product. A product page reads one document. Never joins (see [catalog model](https://docs.rual.nl/blueprints/storage-examples#product-catalog)). - Price in cents with `default_divide_by = 100`: no float rounding bugs. - Everything stored lowercase so case-sensitive search just works. ## The Storefront - **Category page**: term filter on `category` + `status = active`, sorted by name, paginated with `limit`/`offset`. - **Search box**: [`query_bool_simple_query_string_field`](https://docs.rual.nl/block-types/query/query_bool_simple_query_string_field) on `name` wrapped in [`query_bool_must`](https://docs.rual.nl/block-types/query/query_bool_must), with the active-filter in [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) via `query_and`. - **Product page**: fetch by `sku` with [`function_search_single_result`](https://docs.rual.nl/block-types/storage/function_search_single_result); render variants from the embedded array, disabling out-of-stock sizes in the UI. - **Autocomplete**: prefix/wildcard query on `name` behind a small API. The pattern in [search optimization](https://docs.rual.nl/blueprints/storage-examples#search-optimization). ## Stock Without Double-Selling - Checkout decrements stock with [`mutations_increment_by_field`](https://docs.rual.nl/block-types/mutations/mutations_increment_by_field) (`-quantity`) inside `function_update_document`: mutation transactions process sequentially, so two checkouts can't overwrite each other. - Guard the sale: only decrement when `stock >= quantity`; on the failure path, hold the order as `backorder` instead of rejecting the customer. - Restock is the same increment with a positive value. One block, both directions. ## Admin Side - Product management pages scoped `*loggedin` + `catalog_manager` custom scope. See [User Authentication](https://docs.rual.nl/tutorials/user-authentication). - Price/name changes go through [`function_update_document_mutations`](https://docs.rual.nl/block-types/storage/function_update_document_mutations); images via the asset pipeline in [File Upload & Processing](https://docs.rual.nl/tutorials/file-upload-processing). - Discontinuing is a status change, not a delete: old orders keep their embedded snapshots intact. ## Next Steps - [Storage Examples](https://docs.rual.nl/blueprints/storage-examples): The catalog and order models in full. - [Inventory Tracker](https://docs.rual.nl/examples/inventory-tracker): Stock-focused variant of this example. - [Creating a REST API](https://docs.rual.nl/tutorials/rest-api): Expose the catalog to other systems. ## Frequently asked **How do I model product variants in RUAL storage?** Keep variants inside the product document as an embedded array, so a product page reads one document and never joins. Render the variants from that array and disable out-of-stock sizes in the UI. **How do I prevent double-selling stock in RUAL?** Decrement stock with mutations_increment_by_field inside function_update_document, guarded by a stock >= quantity check. Mutation transactions process sequentially per document, so two checkouts cannot overwrite each other, and on the failure path you hold the order as backorder instead of rejecting the customer. **Why store prices in cents in a RUAL catalog?** Storing the price in cents with default_divide_by = 100 avoids float rounding bugs, and the display divides it back. The same model stores names in lowercase so case-sensitive search just works. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Example: User Management System · RUAL Documentation" description: "Profiles alongside cluster users, teams and role changes." canonical: https://docs.rual.nl/examples/user-management language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Example: User Management System Invite, role, suspend, and audit the people using your app. A complete admin section built on cluster accounts and a profiles storage. Cluster accounts handle sign-in and tokens; your app adds profiles, roles, and lifecycle. This example builds the full admin section on top of both. ## Accounts vs Profiles - **Cluster user**: credentials, scopes, tokens; created by an admin via [User Access Management](https://docs.rual.nl/cluster/user-access-management) or the users API. - **Profile document**: your app's data about the person: display name, team, role, settings; stored in a `profiles` storage keyed by the user's guid. ``` { "_meta": { "cms": 1780531200274, "created": 1780531200, "entity": 1, "expiry": -1, "guid": "b9ee6a631b24e78d1aa48aef0dc067fff7bfbacd24a56ef4ed1f08e537d48b6b", "removed": 0, "ums": 1782166695586, "update_hash": "2c6a91e4f7530bd8146ae9c30fb5724d81ea3f06", "updated": 1782166695 }, "display_name": "sami b", "preferences": { "digest": "weekly", "theme": "dark" }, "role": "manager", "status": "active", "team": "engineering", "user_guid": "b9ee6a631b24e78d1aa48aef0dc067fff7bfbacd24a56ef4ed1f08e537d48b6bd02" } ``` ## The Lifecycle - **Invite**: admin creates the cluster user (+ their scopes), then a flow creates the profile with `status: invited` and a welcome email ([Email System Setup](https://docs.rual.nl/tutorials/email-system)). - **First login**. A flow on the member home page sees `status: invited`, flips it to `active`, and shows the onboarding checklist. - **Role changes**: update the profile's `role` for app behavior, and the user's `custom_scopes` for API/page gating. Keep both in one admin flow so they never drift. - **Suspend**: `status: suspended` on the profile blocks app screens; removing the user's scopes blocks the APIs. Account deletion is the last resort, suspended keeps history. ## The Admin Pages - **Directory**: search profiles with filters (team, role, status). The list pattern from [First CRUD App](https://docs.rual.nl/getting-started/first-crud-app), scoped `*loggedin` + `people_admin`. - **Detail page**: profile fields editable, scopes visible, activity summary (last login from the cluster's user activity). - **Self-service page**: members edit only their own profile: fetched with `user_current` → [`function_get_document`](https://docs.rual.nl/block-types/storage/function_get_document) on `user_guid`. See [knowing who's calling](https://docs.rual.nl/tutorials/user-authentication#knowing-who-calls). ## Audit Trail - Every admin action (invite, role change, suspend) writes an event document: actor (from `user_current`), target, before/after. The activity-log model from [Storage Examples](https://docs.rual.nl/blueprints/storage-examples#activity-logs). - Review per-user history with a term query on `target_guid`, sorted by `_meta.created`. ## Next Steps - [User Roles Explained](https://docs.rual.nl/cluster/user-roles-explained): The scopes this example assigns. - [Building User Authentication](https://docs.rual.nl/tutorials/user-authentication): The auth system underneath. - [Task Management App](https://docs.rual.nl/examples/task-management): Assign work to these users. ## Frequently asked **What is the difference between a cluster user and a profile in RUAL?** The cluster user holds credentials, scopes and tokens, created by an admin through User Access Management or the users API. The profile document is your app's data about the person, such as display name, team, role and settings, stored in a profiles storage keyed by the user's guid. **How do I suspend a user in RUAL without deleting their history?** Set status: suspended on the profile to block app screens, and remove the user's scopes to block the APIs. Account deletion is the last resort, because suspending keeps the history. **How do I keep an audit trail of admin actions in RUAL?** Write an event document for every admin action, such as an invite, role change or suspend, with the actor from user_current, the target and the before and after values. Review per-user history with a term query on target_guid sorted by _meta.created. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Example: Booking System · RUAL Documentation" description: "Resources, time slots, conflict checks and reminders." canonical: https://docs.rual.nl/examples/booking-system language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Example: Booking System Availability, reservations, reminders, and no double-bookings. A complete scheduling app with the flow that makes conflicts impossible. Booking looks simple until two people grab the same slot at once. This example shows the model, the conflict-proof reservation flow, and the reminder machinery. ## Data Model - **resources**: what gets booked (rooms, tables, staff): name, capacity, opening hours, timezone. - **bookings**. One document per reservation: ``` { "_meta": { "cms": 1782166695442, "created": 1782166695, "entity": 1, "expiry": -1, "guid": "e67919283b4e37c9758326446605ab863bf452cda26f91c2aef74ba31f9a3214", "removed": 0, "ums": 1782170295118, "update_hash": "7b25e0af9163c4d8052ae7f31964bc0d8a5f3c62", "updated": 1782170295 }, "customer_guid": "b9ee6a631b24e78d1aa48aef0dc067fff7bfbacd24a56ef4ed1f08e537d48b6bd02", "customer_name": "sami b", "end": 1782364800, "reminder_sent": false, "resource_guid": "e01ff8643394371a8544d2c9f8a1b3e5d70892c4f6a0b8d1e3f5a7c9b2d4e6f8a0b2", "resource_name": "meeting room 2", "start": 1782361200, "status": "confirmed" } ``` Resource and customer are denormalized onto the booking: calendar views show names without joins (see [Storage Examples](https://docs.rual.nl/blueprints/storage-examples)). ## Checking Availability - Query the resource's bookings overlapping the requested window: `start requested_start`. Two range filters on the same query via [`query_bool_range_field`](https://docs.rual.nl/block-types/query/query_bool_range_field). - No overlaps → slot is free. Overlaps → return the conflicting bookings so the UI can show why. - Generate the day's available slots from opening hours minus confirmed bookings, in the function: not in the page. ## No Double-Booking - The reservation flow first re-checks overlap (never trust the page's earlier check). - Then it creates the booking with a status transition inside one mutation set: mutations process sequentially per document, so the last check and the write can't interleave (see [mutations](https://docs.rual.nl/blueprints/storage#updating-documents)). - On conflict: reply 409 with the overlapping booking. See [error conventions](https://docs.rual.nl/cluster/api-guide#errors). - Cancellation is a status change to `cancelled`. Never a delete. The audit trail and no-show stats survive. ## Reminders - A [`schedule_repeating_event`](https://docs.rual.nl/block-types/events/schedule_repeating_event) runs hourly: query confirmed bookings starting within the next 24h where `reminder_sent = false`. - Send each with the email pattern from [Email System Setup](https://docs.rual.nl/tutorials/email-system), then set `reminder_sent = true`: idempotent even if the run repeats. - Customers manage their own bookings with `user_current` → term query on `customer_guid`. ## Next Steps - [Repeating Events](https://docs.rual.nl/blueprints/repeating-events): The reminder engine's scheduler. - [Creating a REST API](https://docs.rual.nl/tutorials/rest-api): Expose availability to other apps. - [Inventory Tracker](https://docs.rual.nl/examples/inventory-tracker): The same transaction discipline for stock. ## Frequently asked **How do I check whether a time slot is available in RUAL?** Query the resource's bookings overlapping the requested window with two range filters on the same query: start < requested_end AND end > requested_start. No overlaps means the slot is free, and any overlaps are returned so the UI can show why. **How do I prevent double-bookings in RUAL?** Re-check the overlap inside the reservation flow, never trusting the page's earlier check, then create the booking with a status transition inside one mutation set. Mutations process sequentially per document, so the check and the write cannot interleave, and on conflict you reply 409 with the overlapping booking. **How do I send booking reminders in RUAL?** Run a schedule_repeating_event hourly that queries confirmed bookings starting within the next 24 hours where reminder_sent is false, send each one, then set reminder_sent = true. The marker keeps the job idempotent even if a run repeats. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Example: Inventory Tracker · RUAL Documentation" description: "Stock movements as an append-only log with derived levels." canonical: https://docs.rual.nl/examples/inventory-tracker language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Example: Inventory Tracker Items, locations, movements, and low-stock alerts. A complete warehouse tracker where the ledger is always right. Inventory is a ledger problem: current stock is a sum of movements, and every movement must be recorded once, in order. This example builds that ledger with alerts on top. ## Data Model - **items**: sku, name, unit, reorder_level. - **locations**, warehouse/zone/bin. - **movements**. One append-only document per change: ``` { "_meta": { "cms": 1782166695055, "created": 1782166695, "entity": 1, "expiry": -1, "guid": "08193b4a5d6e7f8091a2b3c4d5e6f708192a3b4c5d6e7f8091a2b3c4d5e6f708", "removed": 0, "ums": 1782166695092, "update_hash": "5e0d38c7a1f294b6037ce8a51d9f27b046a3c81e", "updated": 1782166695 }, "actor_guid": "b9ee6a631b24e78d1aa48aef0dc067fff7bfbacd24a56ef4ed1f08e537d48b6bd02", "item_name": "essential cotton t-shirt", "location": "wh-1/zone-a/bin-04", "note": "po-10482", "quantity": 24, "sku": "TSHIRT-BLK-M", "type": "receive" } ``` Movement types: `receive`, `ship`, `adjust`, `transfer` (two linked movements: out + in). Never update movements: corrections are new `adjust` movements (the activity-log principle from [Storage Examples](https://docs.rual.nl/blueprints/storage-examples#activity-logs)). ## Current Stock, Two Ways - **Ledger sum (exact)**: aggregate movements per sku+location when you need the precise figure. - **Counter document (fast)**. A `stock` storage with one doc per sku+location; every movement flow also applies [`mutations_increment_by_field`](https://docs.rual.nl/block-types/mutations/mutations_increment_by_field) to it via `function_update_document` (sequential, race-free. See [mutations](https://docs.rual.nl/blueprints/storage#updating-documents)). Lists and alerts read only counters. Audit job: a daily [`schedule_repeating_event`](https://docs.rual.nl/block-types/events/schedule_repeating_event) recomputes the ledger sum against the counter and logs drift. You will catch a missed movement within a day, not at year-end. ## The Flows - **Receive**: form or API → create movement → increment counter. Idempotent via a source reference (`po-10482`): re-receiving the same reference is skipped. - **Ship**: same, negative increment, with a guard when the counter would drop below zero (hold as `backorder` movement instead of going negative). - **Transfer**. One flow writes the out-movement and in-movement as two linked documents sharing a transfer guid; the counters move in the same flow. - **Count check**. An `adjust` movement with the counted quantity difference; the counter follows the same path as everything else. ## Low-Stock Alerts - After every counter change, compare against the item's `reorder_level`. - Crossing the threshold (and only then: track previous state on the counter to avoid alert spam) creates an alert document and emails the buyer via the [email pattern](https://docs.rual.nl/tutorials/email-system). - Alert page: open alerts sorted by severity, closed with a status mutation once the receive lands. ## Next Steps - [E-Commerce Catalog](https://docs.rual.nl/examples/ecommerce-catalog): Sell through this stock. - [Storage Examples](https://docs.rual.nl/blueprints/storage-examples): The ledger and counter models in full. - [Repeating Events](https://docs.rual.nl/blueprints/repeating-events): The daily drift-check scheduler. ## Frequently asked **How do I track current stock in RUAL?** Two ways: aggregate the movement documents per sku and location when you need the exact figure, or keep a counter document per sku and location that every movement flow updates with mutations_increment_by_field. Lists and alerts read only the counters, and a daily repeating event recomputes the ledger sum against the counter to catch drift. **How do I correct a wrong stock movement in RUAL?** Never update a movement: corrections are new adjust movements. The movements storage is append-only, so the ledger stays a complete, ordered history. **How do I build low-stock alerts in RUAL?** After every counter change, compare against the item's reorder_level, and only when the threshold is crossed create an alert document and email the buyer. Track the previous state on the counter so a stock level that stays low does not spam alerts. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Example: Task Management App · RUAL Documentation" description: "Boards, columns, assignees and due dates." canonical: https://docs.rual.nl/examples/task-management language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Example: Task Management App Boards, assignments, due dates, and activity. A complete team task app assembled from the CRUD tutorial upward. This is the CRUD tutorial grown up: multiple lists, people, and notifications. It reuses the exact flows from [Build Your First CRUD App](https://docs.rual.nl/getting-started/first-crud-app) and shows what to add for a real team tool. ## The Task Document ``` { "_meta": { "cms": 1782166695910, "created": 1782166695, "entity": 1, "expiry": -1, "guid": "f7802a394c5f48da8694374577160bc974cf563deb37a02d3bf085ca420b4325", "removed": 0, "ums": 1782253095327, "update_hash": "a4f8172be5093d6c47ba209fe83c15d074e2b6a9", "updated": 1782253095 }, "assignee_guid": "b9ee6a631b24e78d1aa48aef0dc067fff7bfbacd24a56ef4ed1f08e537d48b6bd02", "assignee_name": "sami b", "board": "engineering", "column": "in-progress", "created_by": "sami b", "due": 1782432000, "labels": [ "ux", "onboarding" ], "priority": "high", "title": "review q3 onboarding flow" } ``` - `column` drives the board view (todo / in-progress / review / done); `board` separates teams. - Assignee is denormalized. The board renders names without joins (see [Storages](https://docs.rual.nl/blueprints/storage)). - Everything searchable (`title`, `labels`) is lowercase at write time. ## The Views - **Board**: one `function_search` per column (term on `board` + `column`, sorted by `priority` then `due`) or one search grouped in the flow with `array` blocks. The dashboard pattern from [Dashboards](https://docs.rual.nl/tutorials/dashboard-charts). - **My tasks**: `user_current` → term on `assignee_guid` + `column != done`, sorted by `due` ascending. - **Detail**: the task + its activity log (term on `task_guid`) on one page. - **Overdue strip**: `column != done` + `due < now` (range query) at the top of every board. ## The Flows - **Create/move**. The CRUD flows from the tutorial, plus writing an activity entry per change (`moved to review by sami b`) via a second create in the same flow. - **Assign**: update `assignee_guid` + `assignee_name` together (never one without the other), then notify the new assignee with the [email pattern](https://docs.rual.nl/tutorials/email-system). - **Daily digest**: [`schedule_repeating_event`](https://docs.rual.nl/block-types/events/schedule_repeating_event): tasks due tomorrow per assignee, one mail each with `reminder_sent` markers to stay idempotent. - **Close**: status to `done` with a timestamp; boards filter it out, stats keep counting it. ## Scaling Notes - Board queries get a `cache_key`: boards are read-heavy and tolerate seconds of staleness (see [caching patterns](https://docs.rual.nl/blueprints/common-patterns#caching-strategies)). - Invalidate the board cache on every task write with a storage-event → Redis delete flow ([cache invalidation](https://docs.rual.nl/blueprints/common-patterns#caching-strategies)). - Past a few thousand open tasks per board, split counters per column onto a stats document instead of counting live. ## Next Steps - [First CRUD App](https://docs.rual.nl/getting-started/first-crud-app): The tutorial this example extends. - [User Management System](https://docs.rual.nl/examples/user-management): The people side of assignments. - [Dashboard with Charts](https://docs.rual.nl/tutorials/dashboard-charts): Aggregate the board into metrics. ## Frequently asked **How do I build a kanban board in RUAL?** Run one function_search per column with a term on board and column, sorted by priority then due, or one search grouped in the flow with array blocks. The column field on the task document drives the board view. **How do I show a user only their own tasks in RUAL?** Take the caller from user_current and query with a term on assignee_guid plus column != done, sorted by due ascending. **How do I keep a busy RUAL task board fast?** Give board queries a cache_key, because boards are read-heavy and tolerate seconds of staleness, and invalidate that cache on every task write with a storage-event to Redis delete flow. Past a few thousand open tasks per board, keep per-column counters on a stats document instead of counting live. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Reference · RUAL Documentation" description: "Lookup material. Short pages you come back to rather than read once." canonical: https://docs.rual.nl/reference language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Reference Lookup material. Short pages you come back to rather than read once. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Block Quick Reference · RUAL Documentation" description: "The blocks you reach for daily, grouped by what they do." canonical: https://docs.rual.nl/reference/block-quick-reference language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Block Quick Reference The 50 blocks that cover 95% of daily work: what each does, and where to read more. Use ⌘K to search this table. ## Pages & UI | Block | Does what | | --- | --- | | [`state_page`](https://docs.rual.nl/block-types/state%20ui/state_page) | Registers a page URL and starts the page flow. | | [`state_render_page`](https://docs.rual.nl/block-types/state%20ui/state_render_page) | Sends the assembled HTML to the browser. | | `state_h1` / text elements | Headings, text, containers on the page. | | `state_form` | Form container element. | | [`state_input_dynamic`](https://docs.rual.nl/block-types/state%20ui/state_input_dynamic) | Typed input field inside a form. | | `state_button` | Button with a click event. | | [`state_form_get`](https://docs.rual.nl/block-types/state%20ui/state_form_get) | Reads the submitted form (`form-object`, `form-error`). | | [`state_advanced_datatable`](https://docs.rual.nl/block-types/state%20ui/state_advanced_datatable) | Pre-styled table fed from a storage. | ## APIs & HTTP | Block | Does what | | --- | --- | | [`on_startup_register_uri_get`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_get) | Registers `GET /api/v1/ `. | | [`on_startup_register_uri_post`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_post) | Registers `POST /api/v1/ ` (webhooks, creates). | | [`httpconnection_current_request`](https://docs.rual.nl/block-types/http%20connection/httpconnection_current_request) | The live request connection object. | | [`httpconnection_get_params`](https://docs.rual.nl/block-types/http%20connection/httpconnection_get_params) | Reads query parameters of the current request. | | [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) | Replies to the caller with JSON + status code. | | `http request` (http connection group) | Calls external APIs. | ## Storage & Queries | Block | Does what | | --- | --- | | `storage` | Selects the storage collection for connected blocks. | | [`function_search`](https://docs.rual.nl/block-types/storage/function_search) | Searches documents (query, limit, offset, cache key). | | [`function_search_single_result`](https://docs.rual.nl/block-types/storage/function_search_single_result) | Searches and returns the first hit only. | | [`function_get_document`](https://docs.rual.nl/block-types/storage/function_get_document) | Reads one document by guid. | | [`function_create_document_from_mutations`](https://docs.rual.nl/block-types/storage/function_create_document_from_mutations) | Creates a document from a mutation set. | | [`function_update_document_mutations`](https://docs.rual.nl/block-types/storage/function_update_document_mutations) | Applies mutations to an existing document. | | [`function_remove_document`](https://docs.rual.nl/block-types/storage/function_remove_document) / [`function_delete_document`](https://docs.rual.nl/block-types/storage/function_delete_document) | Soft-deletes (recoverable) / hard-deletes. | | [`mutations_set_bp_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_set_bp_field_multiple) | Sets several document fields at once. | | [`mutations_set_custom_field`](https://docs.rual.nl/block-types/mutations/mutations_set_custom_field) | Sets one field with a custom value. | | [`mutations_increment_by_field`](https://docs.rual.nl/block-types/mutations/mutations_increment_by_field) | Atomically increments/decrements a number. | | `query_and` / [`query_bool_must`](https://docs.rual.nl/block-types/query/query_bool_must) | Combines query parts (AND / multi-term OR). | | [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields) | Exact match on one field. | | [`query_bool_filter`](https://docs.rual.nl/block-types/query/query_bool_filter) | Attaches filter clauses. | | [`query_bool_simple_query_string_field`](https://docs.rual.nl/block-types/query/query_bool_simple_query_string_field) | Free-text match on a field. | | [`query_bool_range_field`](https://docs.rual.nl/block-types/query/query_bool_range_field) | Date/number range on a field. | | [`query_sort_field`](https://docs.rual.nl/block-types/query/query_sort_field) | Sorts results by a field. | ## Data & Logic | Block | Does what | | --- | --- | | `value_default` / `number_default` | A fixed text / number value. | | [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields) | Builds an object with named fields. | | [`object_update_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_update_fields) | Adds/changes fields on an existing object. | | `array_map` | Transforms every item in an array. | | `branch` | Splits flow into true/false paths on a condition. | | [`condition_not_empty_value`](https://docs.rual.nl/block-types/condition/condition_not_empty_value) | True when the checked value is filled. | | [`function_console_log`](https://docs.rual.nl/block-types/logging/function_console_log) | Writes to the blueprint console. | | [`trigger_custom_function`](https://docs.rual.nl/block-types/globals%2Cfunction%20execution/trigger_custom_function) | Defines/calls a reusable function. | | `function_return` | Returns values from a function. | ## Background & Scheduling | Block | Does what | | --- | --- | | [`function_custom_execute_from_queue`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute_from_queue) | Defers work to the queue (heavy jobs, retries). | | [`schedule_repeating_event`](https://docs.rual.nl/block-types/events/schedule_repeating_event) | Fires a flow on a schedule. | | `storage_event` | Fires on document create/update/save/remove. | | [`function_redis_cache_delete_key`](https://docs.rual.nl/block-types/redis/function_redis_cache_delete_key) | Deletes a Redis cache key (invalidation). | | `user_current` | The calling user's document. | ## Next Steps - [All Blocks](https://docs.rual.nl/block-types): The full 1,600+ block library by group. - [Block Templates](https://docs.rual.nl/block-types/block-templates): These blocks wired into working combinations. - [Finding Blocks](https://docs.rual.nl/block-types/finding-blocks): Search techniques and starter sets. ## Frequently asked **What are the most-used blocks in RUAL?** This page lists the 50 blocks that cover 95% of daily work, grouped into Pages and UI, APIs and HTTP, Storage and Queries, Data and Logic, and Background and Scheduling. Each entry links to the block's full documentation, and you can search the table with ⌘K. **Which RUAL block registers a GET endpoint?** on_startup_register_uri_get registers GET /api/v1/. The POST variant on_startup_register_uri_post is used for webhooks and creates, and httpconnection_set_json replies to the caller with JSON plus a status code. **What is the difference between function_remove_document and function_delete_document in RUAL?** function_remove_document soft-deletes a document, which stays recoverable, while function_delete_document hard-deletes it. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "API Endpoint Reference · RUAL Documentation" description: "Every cluster endpoint with its method, scope and reply shape." canonical: https://docs.rual.nl/reference/api-endpoint-reference language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # API Endpoint Reference The most-used cluster endpoints on one screen, method, path, purpose. Full parameters and examples per endpoint live in the Cluster API reference. Base URL for everything: `https:// /api/v1`. Authenticate with `Authorization: Bearer ` (or the other four accepted locations. See [API Guide](https://docs.rual.nl/cluster/api-guide#authentication)). The complete generated reference with request/response tables is at [Cluster APIs](https://docs.rual.nl/cluster/api). ## Authentication | Method | Path | Purpose | | --- | --- | --- | | POST | `/api/v1/auth/signin` | Sign in, receive an access token | | POST | `/api/v1/auth/signout` | Invalidate the current token | | GET | `/api/v1/info` | Cluster version, flags, host info (public) | ## Blueprints & Blocks | Method | Path | Purpose | | --- | --- | --- | | POST | `/api/v1/blueprints/search` | List/search blueprints | | GET | `/api/v1/blueprints/{guid}` | One blueprint's metadata | | PUT | `/api/v1/blueprints/update/set/active/{guid}` | Activate a blueprint | | PUT | `/api/v1/blueprints/update/set/inactive/{guid}` | Deactivate a blueprint | | GET | `/api/v1/blueprints/deploy/{guid}/actions` | Pending deploy actions | | POST | `/api/v1/blueprintactions/search` | List blocks inside a blueprint | | GET | `/api/v1/blueprintactions/{guid}` | One block's full document | | POST | `/api/v1/blueprintactions` | Create a block | | POST | `/api/v1/blueprintactions/unsaved/{guid}` | Save staged changes | | DELETE | `/api/v1/blueprintactions/unsaved/{guid}` | Discard staged changes | ## Users & Access | Method | Path | Purpose | | --- | --- | --- | | POST | `/api/v1/users/search` | List/search users | | POST | `/api/v1/users/{guid}/reset-login-attempts` | Unlock a locked-out user | | POST | `/api/v1/_system/authentication-groups/search` | List authentication groups | | GET | `/api/v1/_system/userlogs/search` | Read user activity logs | ## Storages, Assets & Logs | Method | Path | Purpose | | --- | --- | --- | | POST | `/api/v1/_system/storages` | Create a storage collection | | POST | `/api/v1/_system/storages/search` | List storage collections | | GET | `/api/v1/_system/storages/{guid}/count` | Document count of a storage | | POST | `/api/v1/_system/assets/stream` | Upload an asset (multipart, ≤500MB) | | POST | `/api/v1/_system/assets/search` | Search assets | | DELETE | `/api/v1/_system/assets/{guid}` | Delete an asset | | POST | `/api/v1/_system/systemlogs/search` | Read cluster system logs | | POST | `/api/v1/_system/auditlogs` | Read the audit log | ## Fields, Templates & Validation | Method | Path | Purpose | | --- | --- | --- | | POST | `/api/v1/blueprintfields` | Create a blueprint field | | POST | `/api/v1/blueprintfields/search` | List blueprint fields | | POST | `/api/v1/blueprinttemplates/search` | List page templates | | PUT | `/api/v1/_system/blueprintvalidation/{guid}/{updatehash}` | Update blueprint validation rules | ## Next Steps - [Cluster API Reference](https://docs.rual.nl/cluster/api): Every endpoint with params, examples, and scopes. - [API Guide](https://docs.rual.nl/cluster/api-guide): Auth, conventions, errors, and workflows. - [Error Code Reference](https://docs.rual.nl/reference/error-code-reference): What each status means and what to check first. ## Frequently asked **What is the base URL of the RUAL cluster API?** Everything hangs under https:///api/v1. You authenticate with an Authorization: Bearer header, or one of the other four accepted locations described in the API Guide. **How do I activate a blueprint through the RUAL API?** PUT /api/v1/blueprints/update/set/active/{guid} activates a blueprint, and the matching set/inactive path deactivates it. GET /api/v1/blueprints/deploy/{guid}/actions lists the pending deploy actions. **Is there a RUAL endpoint to unlock a locked-out user?** Yes. POST /api/v1/users/{guid}/reset-login-attempts unlocks a locked-out user, and GET /api/v1/_system/userlogs/search reads the user activity logs. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "System Settings Quick Reference · RUAL Documentation" description: "System settings in one table, with defaults." canonical: https://docs.rual.nl/reference/system-settings-quickref language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # System Settings Quick Reference All predefined keys on one screen, key, purpose, example. The full guide with secure values and setup walkthroughs is in System Settings Reference. Change any key in RUAL Studio under **System settings**; changes apply immediately. Full explanations, secure-value handling, and setup examples: [System Settings Reference](https://docs.rual.nl/blueprints/system-settings-reference). ## Locale & Formats | Key | Purpose | Example | | --- | --- | --- | | `timezone` | Timezone for `date` blocks | `Europe/Amsterdam` | | `thousand_separator` | Thousands separator | `.` or `,` | | `decimal_separator` | Decimal separator | `,` or `.` | | `default_divide_by` | Divisor for pricing inputs | `100` (cents) | ## Language | Key | Purpose | Example | | --- | --- | --- | | `language_default` | Fallback language | `nl`, `en` | | `language_user_enabled` | Language from user profile | `true` | | `language_query_enabled` | Language from query string | `true` | | `language_browser_enabled` | Language from browser | `true` | ## Dates & Input | Key | Purpose | Example | | --- | --- | --- | | `input_date_format` | Format for date inputs | `YYYY-MM-DD` | | `input_datetime_format` | Format for datetime inputs | `YYYY-MM-DD HH:mm:ss` | | `date_default_format` | Default date rendering | `YYYY-MM-DD HH:mm:ss` | ## Email & Messaging | Key | Purpose | Example | Secure? | | --- | --- | --- | --- | | `email_default_from` | Default from-address | `noreply@yourdomain.com` | No | | `postmark_token` | Postmark server token | `xxxxxxxx-…` | **Yes, always** | | `messagebird_originator` | Originator for 2FA SMS | `YourApp` | No | ## Custom Keys Your own keys (integration tokens, hosts, flags) live here too: named by integration (`stripe_secret_key`, `slack_webhook_url`), secrets marked secure at creation. Read any key in a flow with the `get system setting` block. ## Next Steps - [System Settings Reference](https://docs.rual.nl/blueprints/system-settings-reference): Full explanations, secure values, email setup. - [System Settings](https://docs.rual.nl/blueprints/system-settings): The conceptual overview. ## Frequently asked **Where do I change system settings in RUAL?** In RUAL Studio under System settings, and changes apply immediately. Any key, predefined or custom, can be read from a flow with the get system setting block. **How do I make RUAL handle prices in cents?** Set the default_divide_by key to 100. It is the divisor for pricing inputs, so values are handled as cents while the display shows the divided amount. **Can I store my own integration secrets in RUAL system settings?** Yes. Custom keys such as stripe_secret_key or slack_webhook_url live alongside the predefined keys, named by integration, and secrets are marked secure at creation. **How does RUAL decide which language a visitor sees?** Four keys control it: language_default sets the fallback language, and language_user_enabled, language_query_enabled and language_browser_enabled allow the language to come from the user profile, the query string or the browser. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Component Props Reference · RUAL Documentation" description: "Props, types and defaults for the built-in components." canonical: https://docs.rual.nl/reference/component-props-reference language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Component Props Reference The props every RUAL component understands: uid, uiclass, uistyle, uiattributes, children, plus per-component specifics. Custom React components mounted in RUAL pages receive a standard set of props from the platform, plus the data you wire in yourself. This is the complete contract. Component setup itself: [RUAL Components](https://docs.rual.nl/interfaces/rual-components). ## Universal Props (Every Component) | Prop | Type | Purpose | | --- | --- | --- | | `uid` | string | Unique instance id. Required and must be stable per instance: used for state, keys, and identity. | | `children` | node | Nested content placed by the blueprint (nested state elements). | | `uiclass` | string | CSS classes applied to the component root. The styling hook from the blueprint. | | `uistyle` | string | Inline style string applied to the component root. | | `uiattributes` | string | Extra HTML attributes (e.g. `data-*`, `aria-*`) applied to the root. | ## Per-Component Props | Component | Props | | --- | --- | | `useTranslate` | `translation_key`: key to look up in cluster translations | | `useInput` | `field`. The form field name it binds to | | Iterate contexts (datatables) | `uid`, `index`, `columns`, `row`: per-row data for custom cells | ## Styling Guide - **Default path**: accept `uiclass`/`uistyle`/`uiattributes` and spread them on your root element: blueprint authors then style instances from the canvas without touching your code. - **Keep internals your own**: root-level props are the public API; style everything inside with your component's own classes. - **Respect the platform shell**: pages render inside the cluster's UI: inherit fonts and avoid global resets in component CSS. ## Getting Data In - Wire blueprint values into the component's pins (params appear as props). See [Components](https://docs.rual.nl/interfaces/components). - Fetch cluster data client-side through the RUAL library (`window.RUAL`). See [RUAL Library](https://docs.rual.nl/interfaces/rual-library). - For lists (datatables), read `columns`/`row` in iterate contexts. See [Handle Iterations](https://docs.rual.nl/interfaces/iterations). ## Next Steps - [RUAL Components](https://docs.rual.nl/interfaces/rual-components): The component system these props belong to. - [RUAL Library](https://docs.rual.nl/interfaces/rual-library): Fetching cluster data from components. - [Handle Iterations](https://docs.rual.nl/interfaces/iterations): Row-level rendering in datatables. ## Frequently asked **Which props does every RUAL component receive?** uid, children, uiclass, uistyle and uiattributes. The uid is the unique instance id used for state, keys and identity, and the other three are the styling hooks the blueprint applies to your component's root element. **How does a custom RUAL component get data from the blueprint?** Three ways: wire blueprint values into the component's pins so they appear as props, fetch cluster data client-side through the RUAL library at window.RUAL, or read the columns and row props in iterate contexts such as datatables. **How should I style the inside of a RUAL custom component?** Accept uiclass, uistyle and uiattributes on the root element and keep everything inside on your component's own classes. Pages render inside the cluster's UI shell, so inherit fonts and avoid global resets in your component CSS. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Error Code Reference · RUAL Documentation" description: "What each error code means and what usually causes it." canonical: https://docs.rual.nl/reference/error-code-reference language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Error Code Reference Every status your APIs can return: what it means, and the first thing to check. Print this page and tape it to your monitor. RUAL APIs answer errors as JSON: an HTTP status, a machine-readable code, and a message. For blueprint-side error handling with success/error pins. See [Block Execution](https://docs.rual.nl/blueprints/block-execution#error-handling); for symptom-led fixes. See [Common Issues](https://docs.rual.nl/troubleshooting/common-issues). ## The Error Shape ``` { "error": "DOCUMENT_NOT_FOUND", "message": "The requested document does not exist." } ``` ## 4xx, Caller Side | Code | Meaning | First thing to check | | --- | --- | --- | | `400` | Bad request: malformed body/params/file | Your JSON against the endpoint's parameter table; content-type header. | | `401` | Unauthenticated: missing, expired, or incomplete auth (2FA) | Token present in an accepted location; idle longer than 14 days? 2FA code included? | | `403` | Forbidden: valid token, insufficient scope | The endpoint's required scope vs the user's scopes ([scoping](https://docs.rual.nl/blueprints/remote-access-control)). | | `404` | Endpoint or document not found | Path typos; for custom APIs: blueprint [activated](https://docs.rual.nl/deployment/how-to-deploy)? | | `409` | Conflict: update-hash/version mismatch | Re-read the document; retry with the current `update_hash`. | | `422` | Unprocessable: semantic validation failed | The response body names the failing field/rule. | | `429` | Rate limited | Back off with jittered retries; check per-endpoint limits ([rate limiting](https://docs.rual.nl/cluster/api-guide#rate-limiting)). | ## 5xx, Cluster Side | Code | Meaning | First thing to check | | --- | --- | --- | | `500` | Internal error | Retry once; if persistent, cluster logs in RUAL Studio (All Logs) name the failing block. | ## Common Error Strings | String | Means | | --- | --- | | `MISSING_RUAL_ENDPOINT` | No API is registered at that path/method (typo, wrong method, or inactive blueprint). | | `DOCUMENT_NOT_FOUND` | The guid doesn't exist (or was removed) in that collection. | | `INSUFFICIENT_PERMISSIONS` | The token's user lacks the scope this endpoint requires. | | `FILE_TOO_BIG` | Upload exceeded the 500MB asset limit. | ## Blueprint-Side Errors Blocks fail differently than APIs: every fallible block carries `success` and `error` out-pins. The error pin emits a JSON with the failing block's type, message, and context. The full shape is documented in [Block Execution: Error Handling](https://docs.rual.nl/blueprints/block-execution#error-handling), and debugging technique in [Debugging Blueprints](https://docs.rual.nl/troubleshooting/debugging). ## Next Steps - [API Guide](https://docs.rual.nl/cluster/api-guide): Auth and conventions around these codes. - [Common Issues & Fixes](https://docs.rual.nl/troubleshooting/common-issues): Symptom-led fixes for the frequent failures. - [Debugging Blueprints](https://docs.rual.nl/troubleshooting/debugging): Reading error pins and tracing failures. ## Frequently asked **What does a 403 from a RUAL API mean?** The token is valid but lacks the scope the endpoint requires. Compare the endpoint's required scope against the user's scopes and add the missing one. **What does a 409 from a RUAL API mean?** A conflict: the update_hash or version you sent no longer matches the document. Re-read the document and retry with the current update_hash. **What does the RUAL error DOCUMENT_NOT_FOUND mean?** The guid does not exist, or was removed, in that collection. RUAL APIs answer errors as JSON with a machine-readable code and a message, so callers can branch on the code. **What should I check first on a 429 from a RUAL API?** You are rate limited. Back off with jittered retries and check the per-endpoint limits described in the API Guide. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Location Blocks · RUAL Documentation" description: "Offline geocoding, coordinate maths and points of interest: every geo point, places and map block in one place." canonical: https://docs.rual.nl/reference/location-blocks language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Location Blocks Everything RUAL knows about *where* something is, and *what is near it*. Coordinate maths, address lookup and points of interest, all answered from datasets on the node. These blocks run **offline**, against datasets shipped with the node. No API keys, no rate limits, no per-call cost, and no request leaving the building. Lookups take microseconds, so you can run them inside loops and over whole inventories without thinking about it. | Operation | Time | | --- | --- | | Nearest lookup (kNN), for example `places_nearest` | 6 to 8 µs. A rare category is as fast as a common one. | | Text search, `places_search` | 11 ms | | Family | What it does | | --- | --- | | [`geo_point_*`](#coordinates) | Create, convert and do arithmetic on coordinates. | | [`geo_point_reverse_geocode`, `array_reverse_geocode`, `geo_point_from_city`](#geocoding) | Turn a coordinate into an address, or a place name into a coordinate. | | [`places_*`](#places) | Find points of interest near a coordinate, search them, filter by them. | | [`osmworld_*`](#map-geometry) | Fetch map geometry for rendering. | ## Working With Coordinates A **geo point** is an object with `lat` and `lon`. These blocks make and manipulate one. None of them touch a dataset: they are pure arithmetic. | Block | Does what | Pins | | --- | --- | --- | | [`geo_point_lat_lng_to_point`](https://docs.rual.nl/block-types/geopoint/geo_point_lat_lng_to_point) | Builds a point from two numbers. | in `lat`, `lng` out `geo_point` | | [`geo_point_to_lat_lng`](https://docs.rual.nl/block-types/geopoint/geo_point_to_lat_lng) | Splits a point back into numbers. | in `geo_point` out `latitude`, `longitude` | | [`geo_point_convert_to_object`](https://docs.rual.nl/block-types/geopoint/geo_point_convert_to_object) | Casts a point to a plain object. | in `geo_point` out `object` | | [`geo_point_validate`](https://docs.rual.nl/block-types/geopoint/geo_point_validate) | Answers whether this is actually a usable coordinate. | in `geo_point` out `valid` | | [`geo_point_branch`](https://docs.rual.nl/block-types/geopoint/geo_point_branch) | Picks one of two points on a condition. | in `then`, `else`, `condition` out `geo_point` | | [`geo_point_destination_point`](https://docs.rual.nl/block-types/geopoint/geo_point_destination_point) | Moves a point a given distance along a bearing. | in `geo_point`, `distance` (metres), `current_bearing` (degrees) out `geo_point` | | [`geo_point_random_in_radius`](https://docs.rual.nl/block-types/geopoint/geo_point_random_in_radius) | Returns a random point within a radius. | in `geo_point`, `radius` out `geo_point`, `success`, `error` | | [`geo_point_define_reference`](https://docs.rual.nl/block-types/references/geo_point_define_reference) / [`geo_point_use_reference`](https://docs.rual.nl/block-types/references/geo_point_use_reference) | Names a point once and reuses it elsewhere in the blueprint without rewiring. | see the block pages | Use [`geo_point_validate`](https://docs.rual.nl/block-types/geopoint/geo_point_validate) as a guard before anything that costs work: a missing or malformed point is the most common reason a location flow silently does nothing. Use [`geo_point_random_in_radius`](https://docs.rual.nl/block-types/geopoint/geo_point_random_in_radius) to anonymise a location before showing it publicly, or to generate plausible test data. ## Geocoding: Coordinates To Places, And Back #### A coordinate becomes an address [`geo_point_reverse_geocode`](https://docs.rual.nl/block-types/geopoint/geo_point_reverse_geocode) takes a `geo_point` and returns `street`, `city`, `neighborhood`, `province`, `country`, `country_code`, `success` and `error`. - `neighborhood` is filled only when the point sits inside a larger city. A point in Ittersum returns city *Zwolle* and neighbourhood *Ittersum*, while a point in Siebengewald returns that as the city, because it is a village in its own right rather than part of one. - `street` is the nearest named road and needs the road network dataset loaded. #### Many coordinates at once [`array_reverse_geocode`](https://docs.rual.nl/block-types/array%2Cgeopoint/array_reverse_geocode) enriches a whole array in one call, with no loop. You give it the path to the geo point inside each item (for example `location.location`) and the paths to write each result to. Leave an output key empty to skip that field. | Pin | Purpose | | --- | --- | | `array` | The items to enrich. | | `location_key` | Path to the geo point inside each item. | | `location_fallback_key` | Second path, for the common case where some records carry a verified coordinate and others only a rough one. | | `street_key`, `city_key`, `neighborhood_key`, `province_key`, `country_key`, `country_code_key` | Where to write each resolved field. | | `array` (out) | The enriched items. | | `unresolved` (out) | Items whose point was missing or outside coverage. | ## Always check unresolved Items whose point was missing or outside coverage come back **unchanged**. If you write the array to a store without looking at `unresolved`, you will not notice. #### A place name becomes a coordinate [`geo_point_from_city`](https://docs.rual.nl/block-types/geopoint/geo_point_from_city) takes `city` and `country_code`, and returns `geo_point`, `radius`, `province`, `country`, `country_code`, `success` and `error`. - `radius` is the city's approximate extent in metres, derived from population. Handy for "is this roughly in Breda" without needing a boundary. - Pass `country_code` to disambiguate: there is a Bergen in NL, NO and DE. #### Terrain and infrastructure | Block | Answers | Pins | | --- | --- | --- | | [`geo_point_land_cover`](https://docs.rual.nl/block-types/geopoint/geo_point_land_cover) | What kind of ground is this? | in `geo_point` out `type` (urban, agricultural, forest, grassland, wetland, water, bare, coastal), `code`, `success`, `error` | | [`geo_point_nearby`](https://docs.rual.nl/block-types/geopoint/geo_point_nearby) | Is a road or railway within X metres? | in `geo_point`, `type` (road/rail), `max_distance` out `is_nearby`, `distance`, `nearest_point`, `success`, `error` | | [`geo_point_snap`](https://docs.rual.nl/block-types/geopoint/geo_point_snap) | Pull a point onto the nearest road or railway. | in `geo_point`, `type` out `geo_point`, `distance`, `success`, `error` | On [`geo_point_snap`](https://docs.rual.nl/block-types/geopoint/geo_point_snap), `distance` tells you how far the point moved. A large value means the original point was nowhere near a road, which usually means the input was wrong. ## Places: Points Of Interest The `places_*` blocks are backed by **Overture Maps Places**, a source of roughly **60 million** points of interest in total. The Netherlands is what a node loads by default: roughly 737,000 Dutch shops, schools, stations, restaurants, hospitals and so on. Three ideas explain most of how these blocks behave, and they are worth reading before wiring anything. ## Need another country? Only the Dutch extract is loaded by default, but the source covers the rest of the world. Other countries can be added to your node on request: ask through [support](https://docs.rual.nl/support) and we will build and ship that extract. #### Distances, never yes or no Every spatial block returns **the distance in metres**, not a boolean. That is deliberate: holding the distances, every question is arithmetic you can do yourself, with no second query. ``` within 200 m of a school? nearest_distance_m <= 200 within 317 m? nearest_distance_m <= 317 at least 3 schools within 500 m? results[2].distance_m <= 500 NOT within 500 m of a school? nearest_distance_m > 500 ``` So a radius slider in your UI never re-queries, and no radius is ever baked into stored data where changing it would mean reprocessing everything. #### Categories accept plain Dutch Anywhere a block has a `categories` pin, you can pass group ids or ordinary words. Regular plurals are handled. | Group id | Also accepted | | --- | --- | | `primary_education` | `basisschool`, `basisscholen` | | `bar` | `kroeg`, `kroegen` | | `train_station` | `treinstation`, `station` | `places_categories` lists everything available, and `places_resolve_category` turns one word into a group id. #### Duplicates are collapsed, and the result says so Overture is not deduplicated. Around Breda station the raw data holds three separate records for one station: `Station Breda Centraal`, `Gare de Bréda` and `NS Station Breda`. With `dedup` on (the default) those merge into one result, reported as `merged_count: 3` with `merged_names` listing all three. Meanwhile the bike shed, the car park, the park called *Happy Station*, the Playmobil museum and the neighbourhood council all stay separate: they are different things that happen to share a word. `places_within` reports both `raw_count` and `count`. The gap between them tells you how much duplication the source data carries at that spot. Treat `count` as a tidy search result, not a census: nationally the data holds 1,432 `train_station` records against roughly 400 real stations. #### Finding places | Block | Does what | Pins | | --- | --- | --- | | `places_nearest` | The one you will use most: nearest matches with their distance. | in `geo_point` (required), `categories`, `max_results` (10), `max_distance` (0 = unbounded), `min_confidence`, `dedup` (true) out `results`, `nearest`, `nearest_distance_m`, `count`, `success`, `error` | | `places_within` | Everything inside a radius. | in `geo_point` (required), `radius_m` (required), `categories`, `limit` (100), `sort` (distance/confidence/name), `min_confidence`, `dedup` out `results`, `count`, `raw_count`, `truncated`, `success`, `error` | | `places_in_bbox` | Everything on screen. | in `north_east` (required), `south_west` (required), `categories`, `limit` (500), `min_confidence`, `dedup` out `results`, `count`, `raw_count`, `truncated`, `success`, `error` | | `places_density` | How built up is this area? | in `geo_point` (required), `radius_m` (500), `categories` out `total`, `raw_total`, `per_km2`, `by_group`, `top_groups`, `classification`, `success`, `error` | Results from `places_in_bbox` carry no distance: a box has no centre. Use `places_nearest` when you need distances. On `places_density`, `classification` is `dense_urban`, `urban`, `suburban` or `rural`, driven by the per square kilometre rate, so it does not change when you change the radius. #### Working over a whole inventory These blocks exist so you never write a loop. | Block | Does what | Pins | | --- | --- | --- | | `places_enrich` | Adds nearest-place data to every item in a list, in one call. | in `items` (required), `point_path`, `categories` (required), `max_distance`, `count_radius`, `include`, `output_path` (`places`) out `items`, `count`, `unresolved`, `success`, `error` | | `places_filter` | Keeps or drops items by a places rule, without touching a store. | in `items` (required), `point_path`, `filter` (required), `mode` (keep/remove), `annotate`, `output_path` out `items`, `count`, `removed_count`, `success`, `error` | | `places_facet` | Answers "how many of my objects are near a hospital?" | in `items` (required), `point_path`, `categories`, `radius_m` (500), `filters` out `counts`, `total`, `success`, `error` | Each item that `places_enrich` touches gets one entry per category: ``` { "places": { "hospital": { "count": 0, "distance_m": 1840, "id": "...", "name": "..." }, "primary_education": { "count": 3, "distance_m": 212, "id": "...", "name": "..." } } } ``` Items are copied, never modified in place, and every item comes back whether or not it had a usable coordinate. Check `unresolved` before writing back. `places_facet` counts **items, not places**: an object with three hospitals nearby counts once. #### Search and typed questions `places_search` is one search box over places *and* towns. It takes `query` (required), `near`, `bias_radius_m` (25000), `kinds` (poi/settlement), `categories`, `limit` (10), `typo_tolerance` (2), `min_confidence` and `dedup`, and returns `results`, `best`, `count`, `success` and `error`. - Handles accents (`Bréda` finds `Breda`), typos (`amsterdm centraal`) and partial words as you type. - Towns outrank businesses, so `Breda` gives you the municipality rather than one of the roughly 2,000 companies with Breda in the name. - Pass `near` (your map centre) to prefer nearby results. - Results carry `kind` (`poi` or `settlement`), `subtitle` (`Breda, Noord-Brabant`, or `treinstation · Amsterdam`) and `score`: enough to render a dropdown without a second lookup. `places_parse_query` turns typed Dutch or English into a filter. It takes `query` (required), `language` (auto), `default_radius_m` (500) and `near`, and returns `kind`, `filter`, `negated`, `category_group`, `category_label`, `radius_m`, `min_count`, `geo_point`, `place_name`, `confidence`, `unparsed`, `success` and `error`. | Typed | Parsed as | | --- | --- | | `binnen 300m van een basisschool` | proximity, `primary_education`, 300 m | | `niet bij scholen` | proximity, `education`, negated | | `minimaal 3 kroegen binnen 500m` | proximity, `bar`, 500 m, at least 3 | | `wijk Jordaan` | an area, with coordinates | | `Station Breda` | a place, with coordinates | Parsing is rule-based and deterministic: no model call, no network, the same answer every time. It never guesses silently, so anything it could not read comes back in `unparsed` with a lower `confidence`, and you can fall back to plain search rather than acting on half a filter. `places_match` tests one point against a filter. It takes `geo_point` (required), `filter` (required) and `max_distance`, and returns `matches`, `distance_m`, `count`, `nearest`, `reason`, `success` and `error`. Both `distance_m` and `count` come back **whether or not it matched**, so a radius slider can re-answer from the result without calling again. `reason` is a readable Dutch line you can show the user, for example `3× basisschool binnen 500m; dichtstbij 212m`. #### Categories and dataset info | Block | Does what | Pins | | --- | --- | --- | | `places_categories` | Builds your category dropdown. | in `search`, `group`, `language` (nl) out `groups`, `count`, `success`, `error` | | `places_resolve_category` | Resolves one word to a group, and returns `alternatives` when a word is ambiguous (`school`) rather than picking one. | in `text` (required), `language` out `group`, `label`, `confidence`, `alternatives`, `success`, `error` | | `places_get` | Re-resolves a stored id. | in `id` (required) out `place`, `found`, `success`, `error` | | `places_dataset_info` | Reports what data this node is running. Worth putting on an admin screen. | no inputs out `loaded`, `release`, `vocabulary_version`, `record_count`, `group_count`, `category_count`, `bbox`, `built_at`, `success`, `error` | ## Two blocks want a different out-pin than success Branch `places_get` on `found`, not `success`: an id can vanish between dataset releases, and that is a normal outcome rather than an error. Branch `places_dataset_info` on `loaded`, not `success`: it is the only places block that answers at all when no dataset is loaded. ## Map Geometry [`osmworld_tile`](https://docs.rual.nl/block-types/openstreetmap/osmworld_tile) (in `z`, `x`, `y`, `kinds`) and [`osmworld_bbox`](https://docs.rual.nl/block-types/openstreetmap/osmworld_bbox) (in `bbox`, or `south`/`west`/`north`/`east`, plus `kinds` and `simplify`) both return a GeoJSON FeatureCollection of roads, buildings, water and greenery for drawing a map, on a `geojson` out-pin alongside `count`. ## These two are the exception Unlike everything else on this page, the `osmworld_*` blocks call Overpass **over the network** and cache to disk. They have flow pins and they are not free. Budget for latency and failure the way you would for any outbound call. ## Recipes #### Store how far every object is from a school ``` query objects -> places_enrich items=results, point_path=general.location, categories=basisschool, output_path=places -> mutations write items back ``` Afterwards an ordinary [`query_bool_range_field`](https://docs.rual.nl/block-types/query/query_bool_range_field) on `places.primary_education.distance_m` filters your inventory. No further place lookups, and any radius works, because the distance is stored rather than a flag. #### A search box that understands what was typed ``` places_parse_query query = |- kind = place / area -> geo_point -> move the map |- kind = proximity -> filter -> places_filter over your objects ``` Use `confidence` and `unparsed` to decide whether to trust the filter or fall back to `places_search`. #### How many of my objects are near a hospital? ``` query objects -> places_facet categories = ziekenhuis, apotheek radius_m = 500 ``` `counts` gives you the number per category, and `total` the number of items evaluated. #### Guarding a location flow ``` geo_point_validate -> if not valid, stop -> geo_point_reverse_geocode (address) -> places_nearest (what is around it) ``` ## Reading The Results Honestly The data is good, but it is not a register. These are properties of the source, so meet them here rather than in front of a client. | What | Why it matters | | --- | --- | | Education overcounts by about 6% | 6,704 records against DUO's roughly 6,300 primary schools. That is exactly the filter with legal consequences, so treat it as indicative rather than compliance grade, and check the official register before anything binding. | | Counts are indicative | Deduplication tidies search results, it does not resolve entities. `raw_count` sits next to `count` so the gap stays visible. | | `confidence` is uncalibrated | Median 0.80. Filtering on it may hide coverage as easily as it removes noise, which is why it defaults to no filter. | | `brand` is sparse | Match chains on `name`. Kruidvat reads 1,010 by name and 27 by brand. | | Areas are not official | `wijk Jordaan` resolves against OSM neighbourhood data, not CBS boundaries, and carries no official code. | Every block on this page reports failures on its `error` pin rather than stopping the blueprint, so branch on `success` and handle the code. | Error | Means | | --- | --- | | `PLACES_NOT_AVAILABLE` | No places dataset on this node. | | `GEO_POINT_MISSING_LAT_OR_LON` | The point had no usable coordinate. | | `RADIUS_REQUIRED` | A radius pin was left empty on a block that needs one. | | `CATEGORIES_REQUIRED` | A categories pin was left empty on a block that needs one. | | `FILTER_REQUIRED` | `places_filter` or `places_match` ran without a filter. | | `QUERY_REQUIRED` | `places_search` or `places_parse_query` ran without a query. | ## The Datasets Behind All This The datasets are shared by every entity on the node and loaded from disk at startup. They are gated by `geo.require_data`: with that on (the default) a node **refuses to boot** rather than quietly serving blocks backed by missing data. | File | Powers | Source | | --- | --- | --- | | *(embedded)* | `geo_point_reverse_geocode`, `geo_point_from_city`, `array_reverse_geocode` | GeoNames + OSM | | `network.bin` | `geo_point_nearby`, `geo_point_snap`, the `street` output | OpenStreetMap | | `landcover.bin` | `geo_point_land_cover` | ESA WorldCover | | `places.bin` | every `places_*` block | Overture Maps Places | Because the files are loaded rather than compiled in, refreshing the data needs no new binary: just the file and a restart. You do not have to remember to refresh the places data either. The node checks the published Overture releases at startup and once a day after, and warns when it has fallen three releases behind. ``` places dataset is STALE: running release 2026-01-01.0, latest is 2026-07-22.0 (6 releases behind). Refresh it. ``` ## Next Steps - [Geopoint Blocks](https://docs.rual.nl/block-types/geopoint): The full geopoint group in the block catalog, with every pin. - [Block Quick Reference](https://docs.rual.nl/reference/block-quick-reference): The blocks you reach for daily, grouped by what they do. - [Storages](https://docs.rual.nl/blueprints/storage): Where enriched items are written back, and how to query them. - [Error Code Reference](https://docs.rual.nl/reference/error-code-reference): What each error code means and what usually causes it. ## Frequently asked **Does RUAL geocoding need a Google Maps API key?** No. Reverse geocoding, city lookup, land cover and the places search all run offline against datasets shipped with the node, so there is no API key, no rate limit and no per-call cost. Only the osmworld blocks, which fetch map geometry for rendering, call out over the network. **How do I find out whether a location is near a school in RUAL?** Use places_nearest with the category basisschool. It returns nearest_distance_m, the distance in metres, rather than a yes or no. Compare that number to whatever radius you need, so changing the radius never means running the query again. **How do I add the distance to the nearest school to every document in a RUAL storage?** Run places_enrich over the search result with the path to the geo point inside each item, then write the items back with mutations. The stored distance can then be filtered with an ordinary range query, and it works for any radius because the distance is stored rather than a flag. **Does RUAL have places data for countries other than the Netherlands?** A node loads the Dutch extract by default, roughly 737,000 points of interest, out of a source that holds around 60 million worldwide. Other countries can be added to your node on request: ask through support and the extract is built and shipped to you. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "DNS Zone Blocks · RUAL Documentation" description: "Authoritative DNS from a storage: render a zone whole, check it everywhere, and push only what passed." canonical: https://docs.rual.nl/reference/dns-blocks language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # DNS Zone Blocks Six blocks that run authoritative DNS from a storage: records live as documents, a zone file is rendered from them whole, and nothing reaches a nameserver until every server has checked the exact bytes about to be sent. The design decision that shapes all six blocks is that a zone file is an output, never a document you maintain. Records are ordinary documents in a [storage](https://docs.rual.nl/blueprints/storage), and the file served by your nameservers is rendered from them in one piece, every time. There is no partial edit and no diff to repair, so the operation everybody dreads becomes ordinary: render, check, push, verify. | Block | What it does | Touches a nameserver | | --- | --- | --- | | [`dnszone_import`](https://docs.rual.nl/block-types/dns/dnszone_import) | Turns a Route53 export into records, timers and delegation, ready to store. | No | | [`dnszone_render`](https://docs.rual.nl/block-types/dns/dnszone_render) | Turns stored records into a complete zone file, with a report of every decision it made. | No | | [`dnszone_slot_status`](https://docs.rual.nl/block-types/dns/dnszone_slot_status) | Reports which per-domain records a member of a zone group has answered for, and which nobody has. | No | | [`dnszone_check`](https://docs.rual.nl/block-types/dns/dnszone_check) | Validates the file on every nameserver. Writes nothing, anywhere. | Reads only | | [`dnszone_push`](https://docs.rual.nl/block-types/dns/dnszone_push) | Replaces the live file and reloads, per server. Only reachable once the check passed everywhere. | Yes, writes | | [`dnszone_verify`](https://docs.rual.nl/block-types/dns/dnszone_verify) | Asks every server which version it is actually serving, and whether they agree. | Reads only | ## One Record Shape, Everywhere What the import block produces is what a storage holds and what the render blocks read. A record carries its owner, type, time to live and values, plus a few fields describing its own disposition inside a group. **Owners are stored relative to the domain**: `@`, `www`, `*`, `mail._domainkey`. That is what makes one record set serve several domains, because `www` renders correctly under every one of them without a single value being rewritten. Storing a fully qualified owner would mean editing it per domain, which is precisely the manual state this design exists to remove. **The start-of-authority record and the domain's own nameserver set are not records.** They come back on their own outputs and every rendered zone carries its own, so there is nothing in the editable set inviting somebody to hand-edit a serial number. ## Importing From Route53 [`dnszone_import`](https://docs.rual.nl/block-types/dns/dnszone_import) reads a Route53 export and only a Route53 export. It accepts the AWS document, a bare array, or either as pasted JSON text, so reading a file and pasting into a box are not two different flows. Escaped characters are expanded on the way in, so a wildcard arrives as a character your rules can actually match. Two things it refuses to guess: - **Routing policies.** Weighted, latency, geolocation and failover answers have no zone file equivalent at all. Importing one would mean collapsing several answers into one arbitrary answer, so the block refuses and names the records it refused. - **The serial number.** It is discarded on import, because the serial is stamped at push time. Storing the old one would create a second answer to a question that already has one, and the two would disagree the first time anybody edited the stored copy. Record order is preserved exactly as the export gave it, which is what makes the same stored state render to the same bytes every time. > **Slot annotations do not survive an import** The markers that bind a record to a per-domain slot are not a Route53 concept, so an import leaves them empty on every record. Apply them from your group definition after importing, not by editing the export first. ## Zone Groups: One Record Set, Several Domains A zone group is one set of records with several domains rendered from it. You edit one domain and the others are rendered from the same records. This is deliberately not three zones with a synchronisation feature: synchronisation is a process, a process can fail halfway, and then two of your three domains are correct. One record set with three renders has no state that can drift. Setting the base domain on [`dnszone_render`](https://docs.rual.nl/block-types/dns/dnszone_render) selects that shape. Every value of every base record then takes one of four dispositions, from the first matching rule you gave it. | Disposition | What happens | | --- | --- | | Verbatim | The default. The same value appears under the new domain. | | Rewrite | A value that names the base domain is re-aimed at this member's. Opt-in, reported, and refused outright when it would change nothing. | | Per-domain | The base value is not copied at all. The record becomes a named slot this domain fills from its own records. | | Note | Copied verbatim and flagged for a human to look at. | **A rewrite that changes nothing is a hard failure, not a shrug.** Providers are inconsistent about whether a stored target ends in a dot, so a rule written against the wrong form silently leaves the member domain pointing at the base domain. That is a real bug that ships quietly, and the only reason to catch it is that the renderer refuses rather than accepting a rewrite with no effect. **Per-domain slots are not an optional refinement.** Certificate validation records are issued per domain, so copying the base domain's publishes a validation record for a certificate nobody requested there. Mail signing keys are minted per sending domain, so a copied key signs nothing and collides with the real one the day that domain starts sending. Ownership verification tokens prove nothing under another domain. Each of those has to be answered by the domain itself or explicitly marked as never applicable. ## Filled, Not Applicable, and Empty Are Three States [`dnszone_slot_status`](https://docs.rual.nl/block-types/dns/dnszone_slot_status) reports the same picture without rendering anything, for an overview screen that has to show the gaps before anyone renders. | State | Meaning | | --- | --- | | Filled | This domain carries its own record for the slot. | | Not applicable | Somebody recorded why this domain will never have one. | | Empty | Nobody has looked at it yet. | The last two are deliberately different states. A domain that sends no mail will never have a mail signing key, and writing that down with its reason is finished work. An empty slot is unfinished work, the render reports itself incomplete, and in strict mode it refuses to render at all. An unexplained gap must never produce a file that looks finished. ## Rendering the File Without a base domain, the same render block produces a plain zone file: the same records, the same timers and the same time-to-live values the zone holds today, so your servers are a drop-in addition while the old delegation is still live. Two things are the platform's own. - **The serial is a placeholder that the push stamps** with the time it was pushed. That is monotonic without anybody having to remember yesterday's number, which is the standing failure mode of the date-plus-counter convention. - **The nameserver set is the zone's existing servers plus yours.** Both sets are genuinely authoritative and serve identical data. Nothing here touches your registrar delegation, which stays a decision you make separately. **Alias records are the one impure part, and the block is loud about it.** A zone file can only hold the addresses an alias resolves to at this moment, and those go stale silently when the target moves. So flattening is opt-in, and every flattened record comes back on its own output with the value it resolved to, on every single render. A target that does not resolve is left out of the file and said so in a comment rather than guessed at. ## Nothing Reaches a Server Without Passing the Gate The render blocks cannot push, and [`dnszone_push`](https://docs.rual.nl/block-types/dns/dnszone_push) will not run ungated. It requires the results from [`dnszone_check`](https://docs.rual.nl/block-types/dns/dnszone_check) and refuses unless every server in its own list has a passing check whose fingerprint matches a fresh hash of the exact bytes about to be sent. That is two guarantees in one rule. A validation failure on any single server leaves every server's live file untouched, including the ones that would have accepted it. And a flow cannot check one file and push a different one, because the bytes are hashed again at the moment of pushing. It also refuses when the render reported itself incomplete. A zone with a per-domain slot nobody answered for parses perfectly and is missing a record, which no validator can see. Past the gate it is deliberately not atomic. The servers are independent, each validates and swaps on its own, and one that rejects the file keeps serving what it had. Follow every push with [`dnszone_verify`](https://docs.rual.nl/block-types/dns/dnszone_verify): disagreement between servers is the only failure that is invisible from the stored copy. > **Server connections carry paths, never keys** A value on a pin is echoed in simulation responses, captured in replay records and rendered in the studio value inspector, so a private key on one pin reaches all three. The blocks refuse a server entry carrying a key, password, passphrase or secret: they take the path to the key file instead. The same reasoning behind [service providers](https://docs.rual.nl/blueprints/service-providers), applied to a server connection. > **Never simulate a push** Simulation holds back storage writes. It does not hold back outbound integrations, so a simulated push really connects to your nameservers and really replaces their zone files. Simulate the render and the check freely; run the push for real, on purpose. ## The Normal Flow, End to End - Import once, into a storage. From then on the storage is the source of truth and the export is history. - Render the file for the domain. Read the report: it names every rewrite, every flattened alias with the value it resolved to, and every gap. - If any slot is empty, fill it or record why it will never apply. Do not push past it. - Check the rendered bytes on every server. - Push, which will only proceed if every check passed on those exact bytes. - Verify. Servers that disagree are the failure you cannot see from here. ## Next Steps - [All DNS Blocks](https://docs.rual.nl/block-types/dns): Full pin reference for each of the six blocks. - [Storages](https://docs.rual.nl/blueprints/storage): The document model the records live in. - [Service Providers](https://docs.rual.nl/blueprints/service-providers): Where credentials live when a flow has to reach something outside the cluster. ## Frequently asked **How does RUAL manage DNS zones?** Records live as documents in a storage and the zone file is rendered from them whole, every time. There is no partial edit and no diff to repair. The normal cycle is import once, render, check the exact bytes on every nameserver, push only what passed, then verify that the servers agree. **Can a RUAL DNS push go out without being checked?** No. The push block requires the check results and refuses unless every nameserver in its list has a passing check whose fingerprint matches a fresh hash of the bytes about to be sent. A failure anywhere leaves every server's live file untouched, and a flow cannot check one file and push a different one. **Is it safe to simulate a DNS push in RUAL?** No. Simulation holds back storage writes but does not hold back outbound integrations, so a simulated push really connects to your nameservers and really replaces their zone files. Simulate the render and the check freely, and run the push deliberately. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Architecture · RUAL Documentation" description: "Decisions you make once per project, and the reasoning behind them." canonical: https://docs.rual.nl/architecture language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Architecture Decisions you make once per project, and the reasoning behind them. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Choosing Your Approach · RUAL Documentation" description: "Blueprint, custom component or external service: which fits which problem." canonical: https://docs.rual.nl/architecture/choosing-approaches language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Choosing Your Approach Blueprints or custom code? Storage or an external database? Real-time or polling? The honest trade-offs, decided per situation. RUAL rarely forces a single way to build something. This guide lays out the recurring architecture choices and gives a default answer for each, so you decide once and move on. ## Blueprints vs Custom React | Situation | Choose | Why | | --- | --- | --- | | CRUD apps, admin panels, forms, dashboards | **Blueprints** | State blocks render pages server-side; zero frontend code, instant auth and storage. | | Highly custom interactions (drag & drop builders, rich editors, games) | **Custom React component** | The component system lets you mount your own React inside a RUAL page. See [Components](https://docs.rual.nl/interfaces/components). | | Data-heavy logic, integrations, scheduled work | **Blueprints (functions)** | Server-side, close to storage, queueable. | | A pixel-perfect marketing site | **Custom frontend on the APIs** | Use RUAL as headless backend: cluster APIs + your own app. See [API Guide](https://docs.rual.nl/cluster/api-guide). | Rule of thumb: blueprint until a specific UI requirement forces you out, then drop to a React component *inside* the blueprint: not to a separate app. ## RUAL Storage vs External Databases | Situation | Choose | | --- | --- | | Application data, documents, orders, profiles, logs | **RUAL storage**: schema-less, searched by query blocks, zero ops. See [Storages](https://docs.rual.nl/blueprints/storage). | | Legacy data that already lives elsewhere | **External via API**: wrap the system with `http request` blocks or custom APIs in front of it. | | Heavy relational reporting across many entities | **Hybrid**: operational data in RUAL storage, periodic exports to your BI warehouse. | | Millions of writes/hour, strict relational integrity | **External database**, with RUAL as the application layer on top. | Storage is schema-less and denormalized by design. If you catch yourself wishing for joins, model *for the read* instead: embed what pages display (see [Storage Examples](https://docs.rual.nl/blueprints/storage-examples)). ## Real-Time vs Polling | Situation | Choose | | --- | --- | | Lists that should update as data changes (live dashboards, shared boards) | **Real-time search**. The page subscribes and re-renders on change. | | External systems without webhooks | **Polling**. A [repeating event](https://docs.rual.nl/blueprints/repeating-events) that fetches on an interval and writes to storage. | | Busy data with many concurrent readers | **Cache + polling**: real-time on hot data costs more than it saves; cache keys and short intervals usually win. See [Common Pitfalls](https://docs.rual.nl/blueprints/common-pitfalls). | ## One Big Blueprint vs Many Small Ones - **Default: one blueprint per component**. A page, an API endpoint, or a named function set. It saves, deploys, and reviews as a unit. - **Share logic with functions, not more blocks**. A public [`trigger_custom_function`](https://docs.rual.nl/block-types/globals%2Cfunction%20execution/trigger_custom_function) is the reuse boundary (see [Best Practices](https://docs.rual.nl/blueprints/best-practices)). - **Split when**: a blueprint stops fitting on your mental screen, two people edit it at once, or a part needs its own deploy cadence. - **Keep together when**: blocks only make sense as one transaction or one page: splitting adds indirection for nothing. ## Decision Summary - UI: blueprint pages first; React component inside the page when needed; external frontend last. - Data: RUAL storage first; external only for legacy or extreme scale. - Freshness: real-time for shared views, polling+cache for volume. - Structure: many small blueprints joined by functions; split on people and deploy cadence, not on line count. ## Next Steps - [Version Management](https://docs.rual.nl/architecture/version-guide): Stable vs development cores, upgrades, and breaking changes. - [Core Concepts](https://docs.rual.nl/getting-started/core-concepts): The platform model these decisions build on. - [Blueprint Best Practices](https://docs.rual.nl/blueprints/best-practices): Applying these choices on a real cluster. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Version Management · RUAL Documentation" description: "Release channels, safe upgrade routines and where breaking changes are documented." canonical: https://docs.rual.nl/architecture/version-guide language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Version Management RUAL Core versions, stable vs development releases, how to upgrade safely, and where breaking changes live. RUAL Core evolves continuously. Clusters run on a specific core version, and you choose when to move to a newer one. This guide explains the release model and a safe upgrade routine. The full per-version history lives in [Core versions](https://docs.rual.nl/core-versions). ## Release Channels | Channel | What it is | Who should run it | | --- | --- | --- | | **Latest stable** ( latest ) | The current production-ready core. Receives only fixes and compatible improvements. | Production clusters, always. | | **Development** ( development ) | The next core in progress. New blocks and changes land here first and may still shift. | Test clusters, early adopters, preparing for the next stable. | | **Older stables** | Previous stable lines, kept for reference in the release notes. | Clusters mid-migration. | Version numbers read as `major.minor.patch` (e.g. 14.0.3): majors and minors bring new blocks and behavior, patches bring fixes. The sidebar's version card always shows the latest stable. ## A Safe Upgrade Routine - **Read the release notes** for every version between yours and the target. See [Core versions](https://docs.rual.nl/core-versions). Focus on "new and changed blocks" and anything marked deprecated. - **Check your blueprints for affected blocks**: in RUAL Studio, the Deprecated Actions view lists blocks in your cluster that changed behavior or are deprecated. - **Upgrade a copy first**. Test on a development cluster or during a low-traffic window. Your blueprints, documents, and settings carry over; the runtime changes underneath them. - **Smoke-test the critical paths**: your pages, your busiest APIs, sign-in, and any integrations (tokens, webhooks, email). - **Watch the logs** for the first hours after upgrading (Studio → All Logs). ## Breaking Changes & Deprecations Breaking changes are rare and always documented in the version's release notes: look for warning callouts at the top of a version page (for example, an API being replaced or a property renamed). Deprecated blocks keep working for a long time; the block page for a deprecated block shows the version it was deprecated in and usually its successor. Migration steps, when needed, are written into the same notes. ## Which Version Should You Use? - **New cluster, production**, latest stable. There is no reason to start older. - **Running production**: latest stable, on your own schedule. Read the notes before each jump. - **Evaluating an upcoming feature**. A development core on a test cluster. Never on customer-facing production. - **Enterprise Cloud**: upgrades are coordinated with RUAL support on a schedule that fits your SLAs. ## Next Steps - [Core Versions](https://docs.rual.nl/core-versions): Release notes for every core version. - [How to Deploy](https://docs.rual.nl/deployment/how-to-deploy): Getting blueprint changes to production. - [Choosing Your Approach](https://docs.rual.nl/architecture/choosing-approaches): The architecture decisions this versioning supports. ## Frequently asked **Which RUAL Core version should I run in production?** Run the latest stable release in production. The development release is where new work lands first and can still change, so use it to try upcoming features, not to serve users. **How do I upgrade RUAL Core safely?** Read the release notes for every version between your current one and the target, watch the warning callouts at the top for breaking changes, and check the deprecated block list against your blueprints through Deprecated Actions in RUAL Studio. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "RUAL Core Nano · RUAL Documentation" description: "The whole platform on one machine: SQLite instead of Elasticsearch, one box instead of a stack, and when to choose it over a cluster." canonical: https://docs.rual.nl/architecture/core-nano language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # RUAL Core Nano The whole platform on one machine. RUAL Core on embedded SQLite instead of Elasticsearch, one box instead of an assembled stack, and the same blueprints you already write. RUAL Core Nano is not a different product. It is RUAL Core configured for a single machine: one application on one box you own, or one node inside the building it automates. The document backend swaps from Elasticsearch to embedded SQLite, and a different set of blocks is enabled. Everything else is unchanged: blueprints, state pages, endpoints, the queue and auth behave exactly as they do on a cluster. The point is latency and independence. A page render, an API call or a sensor two metres from the node should not need a round trip to servers in another country, and the application should not stop working because somebody's internet did. ## One Box Instead of a Stack Every capability a small application needs is usually assembled from a separate service, each with its own account, its own configuration and its own model of a user. In Nano each one is part of the core: | Capability | Typically assembled from | In Nano | | --- | --- | --- | | Document storage | a hosted Postgres or document service | embedded SQLite, same storage blocks | | Search and filtering | the database, or Elasticsearch alongside it | the SQLite engine, see the search note below | | Auth, sessions, tokens | an auth SaaS, or the database's auth add-on | built in: users, roles, scopes, tokens | | File storage | object storage, or the vendor's storage add-on | the asset store, on the same disk | | HTTP endpoints | serverless functions on a host | blueprint endpoints | | Background work | a queue service or a worker process | the built-in queue | | Scheduled work | a cron service | repeating events | | Frontend | a separately hosted frontend app | state pages and custom React components | | Deploys | a git-connected build pipeline | the deployment flow | | Backups | configured per service, separately | one SQLite file and one asset directory | ## What one box buys One process to run, one configuration file, one restore path. Nothing needs wiring between services that each keep their own model of a user, because users, data and endpoints already live in one place. The resource footprint is fixed and sits on hardware you own, rather than being metered per invocation. And the node keeps working with the internet down. ## What you give up No CDN or edge presence, so pages are served from wherever the machine sits. No autoscaling: the machine is the capacity. One machine is one point of failure. And SQLite search differs from Elasticsearch exactly where search gets hard: free text. The limits section below has the details. ## When to Choose Nano Over a Cluster | Situation | Choose | | --- | --- | | One application on one machine, or one building to run | **Nano** | | Data stays on premises, or the thing must keep working with the internet down | **Nano** | | A public service with users everywhere | **Cluster** | | Load that spikes, or grows past what one machine serves | **Cluster** | | Uptime that must survive a single machine failing | **Cluster** | Both shapes run the same blueprints, so starting on a Nano does not paint you into a corner. The growth path is below. ## Minimum Specifications These are the numbers to buy against, not the numbers where it technically boots. | Component | Minimum | Recommended | Why this number | | --- | --- | --- | --- | | CPU | 4 cores, ARM64 or x86-64 | Apple M2 / M4, or a modern 6-core x86 | Blueprint execution is the load, and it is bursty rather than sustained. Four cores keeps the queue workers and a page render from contending, with room left for whatever runs beside the core. | | RAM | **8 GB** | **16 GB** | The core itself sits around 300–500 MB. What consumes the rest is everything beside it: SQLite in WAL mode wants page cache, and Redis holds the hot document cache. 8 GB runs the node on its own; 16 GB is the comfortable number once local home automation adds its daemons, or the geo datasets are loaded. | | Storage | 128 GB SSD | 512 GB SSD | SSD is not negotiable: SQLite's write path is fsync-bound, and on spinning or SD-card storage that is the difference between a responsive node and a visibly slow one. SD cards additionally wear out under WAL churn. | | Network | Wired Ethernet | Wired Ethernet | The node is a server: clients, integrations and any local devices all reach it over the network. A server on WiFi puts a retry loop between every request and its destination. | | Power | | Small UPS | A node that loses power mid-write does not always come back cleanly. A small UPS is the difference between a clean shutdown and a fsck. | > **Radios change the USB story** USB only matters once radios are attached for local home automation: the [tutorial](https://docs.rual.nl/tutorials/local-home-automation) covers ports, hubs and the extension cables. #### Reference Builds | Machine | Verdict | | --- | --- | | **Mac Mini M2, 16 GB** | The comfortable default. Silent, low power, and enough headroom to run the home-automation daemons in Docker alongside the node. Two USB-A ports, which matters once radios are attached. | | **Mac Mini M4, 16 GB** | Same role, faster chip. The M4 has no USB-A ports (front USB-C, rear Thunderbolt), so radios hang off a hub; see the tutorial. | | **Mac Mini M2, 8 GB** | Fine for the node and a modest blueprint set. Tight once you add home-automation daemons. | | **Intel NUC / mini-PC, 16 GB, NVMe** | Equivalent, usually cheaper, usually louder. | | **Raspberry Pi 5, 8 GB, NVMe HAT** | Workable for a small installation. Use NVMe, not an SD card. Expect page renders to be noticeably slower. | | **Raspberry Pi 4 or any SD-card-only board** | Don't. The SD card is the problem, not the CPU. | ## Configuration Start from `configs/nano.example.toml` in the RUAL Core repository. The section that makes a Nano a Nano: ```toml [general] storage_engine = "sqlite" active_entities = [1] file_path = "/var/lib/rual" ``` `storage_engine = "sqlite"` is the whole switch; every other key means what it means on any deployment. Home automation is off by default and configured separately: see [Local Home Automation](https://docs.rual.nl/tutorials/local-home-automation). ## What to Know Before You Commit - **SQLite search is close to Elasticsearch, not identical.** Structured queries are exact; free text is a substring match rather than an analyzed one, and nothing is scored. For a search-heavy application, test the real queries against a Nano before committing to the shape. - **Test the restore, not the backup.** The state that matters is one SQLite file and one asset directory, which makes the backup a single job. Rehearse putting it back onto a fresh machine once, before the box matters. - **Upgrades are not rolling.** There is one process, so a core upgrade is a restart with a short window of downtime. The routine itself is the same as on a cluster: see [Version Management](https://docs.rual.nl/architecture/version-guide). ## Moving to a Cluster Later A Nano is not a dead end. The application layer is identical on both shapes: blueprints, state pages, endpoints and users carry over, because only the storage engine differs. The move is a data job, not a rebuild: stand up the cluster, move the documents and assets across, repoint the URL and deploy the same blueprints. The behaviour worth re-testing afterwards is search, which gains analyzed, scored matching on the cluster and will not behave exactly as SQLite did. ## Home Automation One more reason to pick this shape: a Nano can automate the building it sits in. Zigbee, Thread/Matter, Z-Wave, Philips Hue and Daikin devices are adopted into the same document model and driven by the same blueprint blocks, locally, with no vendor cloud in the loop. It is off by default and costs nothing when disabled. The radios, the daemon stack and the blocks are their own tutorial: [Local Home Automation](https://docs.rual.nl/tutorials/local-home-automation). ## Next Steps - [Local Home Automation](https://docs.rual.nl/tutorials/local-home-automation): Radios, daemons and blocks for automating the building around the node. - [Choosing Your Approach](https://docs.rual.nl/architecture/choosing-approaches): The per-feature architecture decisions that follow this one. - [How to Deploy](https://docs.rual.nl/deployment/how-to-deploy): Getting blueprint changes to production. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Scaling and Automatic Sizing · RUAL Documentation" description: "How storage capacity is sized from what it holds, what the nightly pass changes, and what stays a human decision." canonical: https://docs.rual.nl/architecture/scaling language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Scaling and Automatic Sizing You never choose how much capacity a storage gets. The cluster decides it from what the storage actually holds, revisits the decision every night, and keeps enough copies of your data for the value of the data rather than for a number somebody picked once. A storage is created before it holds anything. At that moment nobody knows whether it will end up holding forty rows or forty million, and asking the person creating it to guess is asking for an answer that is wrong in both directions: too small and it has to be rebuilt later, too large and the cluster pays for room that never fills. So the platform does not ask. What you get instead is described below. None of it needs configuring, and none of it changes how you write a blueprint. It is worth reading anyway, because a handful of design choices on your side make the automatic behaviour work considerably better. ## Decided for you How much capacity a storage gets, how many copies of it are kept, when that decision is revisited, and what happens when a storage grows past the shape it was given. All of it measured rather than guessed, and all of it live: no downtime, no migration, no maintenance page. ## Still yours How many storages you create and what goes in them, whether data has a natural end date, and telling your operator before you load fifty times more than usual. The cluster reacts to growth quickly, but it reacts: it cannot know about tomorrow's import. ## A New Storage Starts at Its Smallest Useful Size Every storage is created at the smallest shape that works, and grows from measurement instead of from prediction. That is not modesty, it is the cheaper direction to be wrong in. Capacity set aside for a storage is paid for whether documents arrive or not, and it is paid for on every machine that holds a copy. An empty storage given room for a million documents costs real memory, real recovery time after a restart, and real work on every single query it answers, for as long as it exists. An application with a hundred storages that were each sized for the biggest they might ever get is an application whose cluster is mostly paying for room nobody is using. Growing a storage that turned out to be popular is a solved problem, covered further down. Reclaiming capacity from a hundred storages that never filled is a much worse afternoon. ## How Many Copies Your Data Keeps Depends on What It Is Every storage is held more than once so that losing a machine does not lose data. How many spare copies is not a single cluster-wide number: it is decided per storage, from what that storage means, because the cost of losing it differs enormously and so does the cost of keeping it. | Kind of data | Spare copies | Reasoning | | --- | --- | --- | | Your business data: the storages you create | One | Survives a machine failing. Two copies is what failover requires; a third is a copy nobody ever reads, paid for on every write. | | Sign-in, sessions, tokens, page definitions | Two | The extra copy is not about durability. It is read on essentially every request, so the copies buy read throughput, and they keep logins working while machines are restarted one by one during an upgrade. | | Change history and audit trails | One, never zero | A history cannot be rebuilt from the current documents, which is exactly what makes it a history. It is kept for years and it is never treated as disposable, whatever the storage it belongs to is used for. | | Caches and derived data | None | Rebuildable by definition. Paying to store several copies of something the cluster can regenerate is the one case where a spare copy buys nothing at all. | | Usage statistics, logs, request history | One | Very high volume, low value per individual record. Worth keeping, not worth keeping three times. | A storage the platform does not otherwise recognise is treated as business data, which is the careful end of that table. Nothing lands in the cache row by accident. > **Small clusters get fewer copies, on purpose** The copy count is capped at what the cluster can actually place: a copy with no machine to live on is not durability, it is a cluster that reports itself permanently unhealthy. On a single-machine cluster the answer is zero spare copies and that is the correct answer, which is one of several reasons a [Nano](https://docs.rual.nl/architecture/core-nano) is backed up rather than replicated. ## A Nightly Pass Keeps the Decision Current A decision made when a storage was empty stops being right the moment it fills, so it is made again every night. Once a day, in a quiet window, the cluster looks at every storage it holds and asks four questions: how large is it, how many documents does it hold, how fast did it grow over the last week and the last month, and how much is it actually read and written. From those answers it does two different things, and the split between them is the important part. | Change | Who does it | Why | | --- | --- | --- | | The number of copies | The nightly pass, immediately | Adding or dropping a copy is a live background operation. Nothing is rewritten, nothing is taken offline, and it is reversible the following night. | | The capacity a storage is spread over | Written down as a recommendation for a person | Reshaping a storage moves its contents. That belongs in a window somebody chose, on a cluster somebody is watching, and never in an automated pass at three in the morning. | Each run leaves a report behind: what it changed, what it recommends, what it skipped and the reason it skipped it. On a new deployment the pass starts in a mode where it reports and changes nothing, so the recommendations can be read against reality for a week or two before anybody lets it act. ## What the Nightly Pass Refuses to Do Something that edits production storage every night is defined at least as much by what it will not do. These are the standing rules, and none of them has an override that runs unattended. | Rule | What it prevents | | --- | --- | | Never fewer than one spare copy of anything that cannot be rebuilt | An automated pass quietly leaving your only copy of something on one machine. One copy is not a backup and it is not failover. | | Never reshapes a storage on its own | Contents being moved without anyone deciding it was a good moment. | | Never runs during business hours | Even adding a copy sends data across the network. It happens while nobody is waiting on a page. | | Never starts while the cluster is unhealthy or already moving data | Piling work onto a cluster that is already recovering, including from the pass's own previous action. | | One storage at a time | A run touching everything at once and making its own effects impossible to read. | | Never more copies than the cluster has machines to hold | Copies that can never be placed, which report as a permanently unhealthy cluster rather than as extra safety. | | A cooldown of about two days after changing a storage, holding the previous value | A storage being changed back and forth night after night, and a change nobody can undo. Every action carries its own undo record. | | Reports before it acts, and reports what it did afterwards | Anything happening to your data that is not written down somewhere you can read. | > **Reductions are allowed when the cluster is merely bruised** Adding a copy needs a fully healthy cluster, because it puts more data onto machines that are already missing something. Removing one is allowed on a cluster that is not fully healthy, because dropping a copy is often the cure: when the unhealthy part is an extra copy that has nowhere to go, waiting for perfect health before removing it would mean waiting forever. ## Growing a Storage Without Taking It Offline When a storage really has outgrown the capacity it was given, it can be grown in place, and the operation is built so that a failure at any point leaves you where you started. - Writes to the storage are held briefly. - A larger copy is built alongside the original. The original stays readable throughout. - Both sides are counted and compared. A mismatch stops everything here, with the original untouched. - The name is moved from the old to the new in one single step, so there is never a moment where the name resolves to nothing and writes fail. - Every other name that pointed at the old storage is carried across with it, so nothing anyone wrote down stops resolving. It runs on request, by a cluster admin, in a window they picked, and it previews by default: you have to explicitly say "do it" for anything to move. It refuses outright on a cluster that is not fully healthy or is already moving data around, and if a step fails before the switchover, the partial work is removed and the original is unblocked. In the one case where the platform cannot be certain what happened, it deletes nothing and says so loudly, because the worst possible response to an uncertain outcome is a confident cleanup. ## Data That Only Grows Is Bounded, Not Enlarged Some data has no natural end: usage statistics, request logs, an event history. Giving those more room is treating the symptom, because whatever you give them they will eventually fill. The platform handles its own such data by cutting it into time buckets with a horizon behind it. Statistics are written into a bucket per month and buckets past the horizon are dropped whole, so history stays bounded at roughly the last year plus the current month rather than growing forever. Dropping a whole month is close to free; deleting a year of records one at a time out of one enormous storage is one of the most expensive things you can ask a cluster to do. The same rule is worth applying to your own high-volume storages. If a storage only ever grows, put a date in the design and decide how far back you actually need, rather than waiting for it to become the largest thing on the cluster. [Document expiry](https://docs.rual.nl/blueprints/storage#document-expiry) covers the per-document version of the same idea. ## The Few Things That Are Still Yours Automatic sizing reacts to what you built. These are the decisions it cannot make for you, roughly in order of how much difference they make. | Do this | Because | | --- | --- | | Keep one storage per kind of thing, not one per customer, tenant or day | This is the single biggest lever you hold. A hundred storages holding forty documents each cost far more than one storage holding four thousand with a field naming the customer, and no amount of automatic sizing recovers that, because the cost is per storage rather than per document. Filter on a field instead. | | Delete storages you stopped using | An empty storage is not free. It is observed, copied and kept healthy exactly like a full one. | | Give data with a natural end an actual end | Sessions, one-time codes, drafts and imports mostly have an obvious expiry. Setting it is cheaper than any capacity decision made later. | | Warn your operator before a large import | The nightly pass measures what happened yesterday. It handles steady growth well and a fifty-fold jump overnight less well, and a heads-up turns that into a scheduled change instead of a surprise. | | Leave the recommendations alone unless you are the one running them | The report is deliberately advisory. Anything it recommends moves data, and it is written for a person with a maintenance window, not for the next deploy. | ## What Automatic Sizing Deliberately Does Not Do - **It does not shrink a storage on its own.** Making a storage smaller is more disruptive than making it larger, so it is always a human decision. - **It does not change your document design.** How many fields a document has and what they contain is a modelling question, and a sizing pass has no business having an opinion about it. See [Storage Examples](https://docs.rual.nl/blueprints/storage-examples). - **It does not tune individual queries.** Capacity is a reliability decision. A slow query is a query problem, and [Common pitfals](https://docs.rual.nl/blueprints/common-pitfals) is the better starting point. - **It never touches another tenant.** The pass is scoped to the entities it is running for, and a recommendation is never derived from anybody else's data. > **None of this applies to a Nano** A single-machine deployment stores its documents locally and keeps no copies on other machines, so there is nothing to size and nothing to balance. Its equivalent of all of the above is a backup schedule: see [RUAL Core Nano](https://docs.rual.nl/architecture/core-nano). ## Next Steps ## Frequently asked **Do I have to size a storage in RUAL?** No. Every storage is created at the smallest shape that works and grows from measurement. A nightly pass looks at how large each storage is, how many documents it holds, how fast it grew over the last week and month, and how much it is read and written, then adjusts the number of copies it keeps. **How many copies of my data does a RUAL cluster keep?** It depends on what the data is. Business storages keep one spare copy, sign-in and session data keeps two because it is read on nearly every request, change history never drops below one, caches keep none because they can be rebuilt, and high volume telemetry keeps one. The count is also capped at what the cluster has machines to hold. **Does RUAL reshape a storage automatically?** No. The nightly pass changes the number of copies itself, because that is a live background operation, and writes down capacity changes as a recommendation for a person. Reshaping a storage moves its contents, so it belongs in a window somebody chose on a cluster somebody is watching. **What is the biggest thing I can do to help a RUAL cluster scale?** Keep one storage per kind of thing rather than one per customer, tenant or day. A hundred storages holding forty documents each cost far more than one storage holding four thousand with a field naming the customer, because the cost is per storage rather than per document. Filter on a field instead. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Troubleshooting · RUAL Documentation" description: "When something does not do what you expect, start here." canonical: https://docs.rual.nl/troubleshooting language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Troubleshooting When something does not do what you expect, start here. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Common Issues & Fixes · RUAL Documentation" description: "Blueprints that do not respond, scope errors, stale caches and their fixes." canonical: https://docs.rual.nl/troubleshooting/common-issues language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Common Issues & Fixes Symptom-led fixes for the most frequent RUAL problems: unsaved blueprints, inactive APIs, stale UI data, slow pages, and connection or token errors. Each issue below is listed by the symptom you observe, followed by the most likely cause and the fix. Every section has an anchor, so you can deep-link directly to a specific issue. For technique-oriented debugging guidance. See [Debugging Blueprints](https://docs.rual.nl/troubleshooting/debugging). ## Blueprint not saving / changes lost **Symptom:** Your changes seem to disappear, or other users never see them. **Likely cause:** The changes were staged but never saved, or they were saved but never deployed. RUAL treats these as three separate states: | State | What it means | | --- | --- | | Staging | Appending `?staging` to a page URL lets you preview unsaved blueprint changes. Nothing is persisted: closing or refreshing without saving loses the work. | | Saved (development) | Clicking `save your changes` (or `Cmd/Ctrl + S`) persists the blueprint in development mode. Since [v13.0.7](https://docs.rual.nl/core-versions/130007), saving does **not** change what runs in production. | | Deployed (production) | Users only see changes after an explicit deploy through the [deployment](https://docs.rual.nl/deployment/how-to-deploy) menu. | **Fix:** Save the blueprint with the `save your changes` button before leaving it, and use `?staging` only for quick previews. If the blueprint is saved but users still see the old behavior, deploy it to production. Note that removed blocks also only disappear permanently once the blueprint is saved. Read more in [Staging and saving a blueprint](https://docs.rual.nl/blueprints/tips-and-tricks#saving-blueprint). ## API not activating / 404 or 401 after activation **Symptom:** A newly created API endpoint returns `404`, or calls return `401` even though the blueprint looks correct. **Likely cause:** The blueprint was never activated, or the caller does not satisfy the endpoint's scopes. All elements in a blueprint: pages, APIs, modals, and events: are inactive by default and only start listening after you press the `Activate` button in the top bar. Keep in mind that activation applies to the development blueprint; production serves the last deployed version, so an endpoint activated but never deployed still 404s for production traffic. **Fix for 404:** Press `Activate`, save the blueprint, and deploy if the calls target production. **Fix for 401/403:** Check the endpoint's scopes via its lock icon: - **Open lock icon**. The endpoint uses the `*public` scope and is accessible to everyone. - **Closed lock icon**. The endpoint is private. With `*loggedin`, only authenticated users get through; with a custom scope, the user's account must hold that scope. Unauthenticated API calls fail, and restricted pages redirect to `/403`. Also verify the request carries a valid `access_token`. See [Adding tokens to your requests](https://docs.rual.nl/cluster/api-guide#token-providing). Full details in [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control). ## State / UI not updating **Symptom:** After creating or updating a document, the page still shows the old data. **Likely cause:** One of three forms of staleness: - **Search indexing delay**. Search queries do not wait for the latest inserts to be indexed, so a freshly created document can be missing from results for a short while. - **Stale Redis cache**. The flow serves cached data that was never invalidated after the write. - **Denormalized copies not synced**: data embedded in other documents was not updated when the source document changed. **Fix:** - When real-time accuracy matters, add the [`disable cache`](https://docs.rual.nl/block-types/query/query_disabled_request_cache) query block so the core waits for pending inserts before searching. Use it sparingly: on high-insert storages it can slow queries down noticeably. - Invalidate or refresh Redis keys from a [storage event](https://docs.rual.nl/blueprints/storage-events) whenever the underlying documents change. - Use storage events to propagate source-document changes to every document that embeds that data. Background in [Real-Time Search Results](https://docs.rual.nl/blueprints/storage#real-time-search) and [Common Pitfalls](https://docs.rual.nl/blueprints/common-pitfalls#disable-cache-query). ## Performance issues **Symptom:** Pages or APIs respond slowly, especially under load, or the cluster shows high memory usage. **Likely cause:** The usual suspects are missing cache, expensive aggregations, and heavy work running inside the request flow: | Cause | Why it hurts | Fix | | --- | --- | --- | | No caching | A single search is cheap, but 300 simultaneous page visits run it 300 times per minute. | Cache frequent search results in Redis with a TTL; invalidate through [storage events](https://docs.rual.nl/blueprints/storage-events). See [Caching with Redis](https://docs.rual.nl/blueprints/storage#caching). | | Aggregation queries on busy storages | Aggregations on heavily populated storages degrade performance and consume excessive memory. | Store counters on the document itself and increment them, or precompute statistics in Redis via [repeating events](https://docs.rual.nl/blueprints/repeating-events). See [aggregation pitfalls](https://docs.rual.nl/blueprints/common-pitfalls#misuse-of-aggregations). | | Heavy work inline | PDF generation, email sending, or batch processing blocks the request until it finishes. | Offload to the [queue](https://docs.rual.nl/blueprints/queue) with [`function_custom_execute_from_queue`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute_from_queue) so work runs in the background. | ## Connection problems (cluster URL, 401s) **Symptom:** API calls never reach your application, or every call returns `401` even though a token is being sent. **Likely cause:** - **Wrong URL**: requests go to RUAL Studio (`rual.at`) instead of your own cluster URL. Studio is the management application; your cluster serves its own APIs and pages on its own URL. - **Expired token**: tokens are valid for 14 days by default and only extend automatically while actively used. An idle token eventually stops working. - **Token sent the wrong way**. The token is not in a place the cluster reads. **Fix:** Confirm you are calling your cluster URL, obtain a fresh `access_token` by logging in through the authentication APIs, and send it in one of the accepted ways: `Authorization: Bearer` header, `x-authtoken` or `x-token` header, an `access_token` cookie, or the `?access_token=` query parameter. See [Authentication](https://docs.rual.nl/cluster/api-guide#authentication) and [Token Expiry](https://docs.rual.nl/cluster/api-guide#token-expiry). ## Still stuck? If none of the above resolves your issue: - **Check the audit log**. Every blueprint modification is logged automatically. Open it via the blueprint's `Options` menu to see whether an unexpected change explains the behavior. See [Audit log navigation](https://docs.rual.nl/troubleshooting/debugging#audit-log). - **Debug the flow step by step**. Use simulation mode and the console as described in [Debugging Blueprints](https://docs.rual.nl/troubleshooting/debugging). - **Contact support**: reach out via [rual.nl/support](https://rual.nl/support). ## Frequently asked **Why does my RUAL API return 404 after I activated it?** An endpoint that still returns 404 after scoping usually was never activated. Press Activate, and deploy it when production traffic should reach it as well. **Why is my RUAL page showing stale data?** Either the state was not updated after the change, or a cached query is still serving the previous result. Check the flow that updates the state, and check the cache key on the search that feeds it. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Debugging Blueprints · RUAL Documentation" description: "The console, the simulation popup, error pins and block revisions." canonical: https://docs.rual.nl/troubleshooting/debugging language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Debugging Blueprints Debug RUAL blueprint flows with simulation mode, the console, production runs, and the audit log. Read error pins and test blocks systematically. This page covers the tools and techniques for finding out *why* a flow misbehaves. If you are looking for a fix to a specific symptom, start with [Common Issues & Fixes](https://docs.rual.nl/troubleshooting/common-issues). For execution semantics: dataflow, parallel branches, and error handling. See [Block Execution](https://docs.rual.nl/blueprints/block-execution). ## Using the console The blueprint console shows execution details for your flows and is often the fastest way to see what a flow actually did. Activate it through the `Options` menu in the top right of your blueprint, then run the flow: for example through simulation mode: and watch the console output to pinpoint where behavior diverges from what you expect. ## Simulating blocks and functions Blocks at the beginning of a flow feature a play icon. Clicking it opens a popup where you can switch between play modes, activate the current mode (default is `simulation`), and adjust simulation values. Each out-pin of the starting block can be given a test value, which is then passed to the connected blocks as the simulation traces through the flow. This makes simulation mode ideal for end-stage debugging: you see the output at every step without touching real traffic. Simulation values must be formatted according to their pin type, or the simulation will not produce the results you expect: | Pin Type | Formatting | | --- | --- | | `value` | `"value"` or `""` | | `number` | `1` | | `condition` | `true` or `false` | | `object` | `{"key": "value"}` or `{}` | | `array` | `["value1", "value2"]` or `[]` | | `date` | `1749321690` (timestamp as number) | | `uiclass` | `"uiclass"` or `""` | | `uistyle` | `{"background": "grey"}` or `{}` | | `uiattributes` | `{"loading": "lazy"}` or `{}` | - [Simulating Functions](https://docs.rual.nl/blueprints/tips-and-tricks#simulation-functions): The full simulation walkthrough in Tips & Tricks. ## Production run vs development view Simulation runs against development data. When a bug only appears with real data, select `Production Run` in the play mode options: it executes the **non-deployed (development)** blueprint against **production data**. This lets you debug the current state of your blueprint with real documents, without deploying anything and without affecting the live production environment. Keep the difference in mind when interpreting results: the development view shows your unsaved/saved working state, while production users run the last [deploy](https://docs.rual.nl/deployment/how-to-deploy). If development works but production does not, the blueprint was probably never deployed. See [Blueprint not saving / changes lost](https://docs.rual.nl/troubleshooting/common-issues#blueprint-not-saving). > **Permission required** The `Production Run` permission can be disabled for specific users in [User Access Management](https://docs.rual.nl/cluster/user-access-management#permissions). If the option is missing from your play mode menu, ask an administrator to enable it for your account. ## Audit log navigation Every blueprint modification is automatically logged: like git commits, but with real-time collaboration. The audit log records who changed what and when, which makes it the first place to check when a flow that used to work suddenly behaves differently. - **Whole blueprint:** open the audit log through the `Options` menu in the top right of the blueprint. - **Single block:** click the block's options () and select `Revisions` to see only the changes affecting that block. ## Reading error messages RUAL has no try/catch mechanism. When a block fails, the flow does not stop: instead, blocks that can fail expose their result through output pins: - `success`. A condition pin that is `true` when the operation succeeded and `false` when it failed. - `error`. A value pin containing the error message when the operation fails. A typical error payload looks like this: ``` { "success": false, "error": "Document not found for the given GUID" } ``` Always connect and inspect these pins on blocks that perform storage operations, HTTP requests, or anything else that can fail. An unconnected `error` pin means failures pass silently. See [Error Handling](https://docs.rual.nl/blueprints/block-execution#error-handling) for the common handling patterns. The table below maps frequent error symptoms to their likely cause and the first thing to check: | Symptom | Likely cause | First thing to check | | --- | --- | --- | | `Document not found for the given GUID` | The GUID fed into the block is empty, wrong, removed, or expired. | Simulate the flow and inspect which value actually reaches the GUID in-pin. | | Flow stops doing anything after a block | The block failed and its `success`/`error` pins are unconnected, so the failure is silent. | Connect the `success` pin to a condition block and log or display the `error` pin. | | Search returns an empty array | Indexing delay, or case-sensitive mismatch between stored and searched values. | Check the casing of the stored value, and retry with the [`disable cache`](https://docs.rual.nl/block-types/query/query_disabled_request_cache) query option. See [real-time search](https://docs.rual.nl/blueprints/storage#real-time-search). | | Function does not execute from another blueprint | The function is private, or the blueprint containing it is not activated. | Check the function's visibility and press `Activate` on the owning blueprint. | | API returns 401 | Missing/expired token, or the caller lacks the endpoint's scope. | See [Connection problems](https://docs.rual.nl/troubleshooting/common-issues#connection-problems) and [Remote Access Control](https://docs.rual.nl/blueprints/remote-access-control). | | Works in development, not for users | Changes saved but never deployed to production. | See [Blueprint not saving / changes lost](https://docs.rual.nl/troubleshooting/common-issues#blueprint-not-saving). | ## Testing blueprints systematically When a flow is too tangled to debug by inspection, work through it in order: - **Isolate**: narrow the problem down to the smallest flow (or function) that still shows the wrong behavior. Use the block `Revisions` to confirm nothing changed unexpectedly. - **Simulate**. Run that flow in simulation mode with controlled values in the correct [pin-type format](#simulation), and trace the output at each step. - **Verify pins**. Check that every block along the path receives the in-pin values you expect and produces the out-pin values the next block needs. Pay special attention to `success`/`error` pins on storage and HTTP blocks. - **Confirm against production data**. If simulation passes but the live flow still fails, repeat the run with `Production Run` to rule out data-dependent issues. - [Common Issues & Fixes](https://docs.rual.nl/troubleshooting/common-issues): Symptom-led fixes for frequent problems: saving, activation, stale data, performance, and connections. - [Block Execution](https://docs.rual.nl/blueprints/block-execution): How blocks execute: dataflow, parallel branches, and error handling. ## Frequently asked **How do I debug a RUAL blueprint?** Use the console to see the executed path and the values that flowed through it, simulate individual blocks and functions, compare the production run against the development view, and read the error pins on blocks that can fail. **Why does my flow fail silently in RUAL?** An unconnected error pin means failures pass without a trace. Wire the error pin of every block that can fail to something that logs or reports it. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Block Templates, Combinations That Work · RUAL Documentation" description: "Copyable block combinations for everyday use cases: API endpoints, forms, validation, file uploads, and email sending." canonical: https://docs.rual.nl/block-types/block-templates language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Block Templates: Combinations That Work Copyable block combinations for everyday use cases: API endpoints, forms, validation, file uploads, and email. Wire them once, reuse the shape forever. Most flows are variations on a handful of proven shapes. These templates give you the exact blocks, in order, with their connections: grab them onto your canvas and adjust the details. Every block type below exists in the current block library; search its name in the editor to add it. For broader recipes with notes and pitfalls. See [Common Blueprint Patterns](https://docs.rual.nl/blueprints/common-patterns). ## 1. API Endpoint The universal shape of every custom API: trigger, work, reply. | Block | Purpose | Connects to | | --- | --- | --- | | [`on_startup_register_uri_get`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_get) | Registers `GET /api/v1/ ` and fires per request. | `function` → trigger's function handler | | [`trigger_custom_function`](https://docs.rual.nl/block-types/globals%2Cfunction%20execution/trigger_custom_function) | The handler function entry point. | `flow` → your work blocks | | [`httpconnection_current_request`](https://docs.rual.nl/block-types/http%20connection/httpconnection_current_request) | The live request connection object. | `connection` → every request/response block that needs it | | Work blocks (e.g. `function_search`, [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields)) | Read/write data, build the payload. | flow chain; data pins as needed | | [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) | Replies with JSON to the caller. | `flow` (end of chain), `connection`, `code` ← `number_default` (200/400/401), `data` ← your payload | Variants: swap the trigger for [`on_startup_register_uri_post`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_post) (body in, e.g. webhooks), add a `branch` for validation splits, or a second [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) on the error path. Full walkthrough: [Build Your First API](https://docs.rual.nl/cluster/api-quickstart). ## 2. Form Handling Collect input on a page and write it to storage on submit. | Block | Purpose | Connects to | | --- | --- | --- | | `state_form` | The form container element. | renders on the page; receives child elements on `content` | | [`state_input_dynamic`](https://docs.rual.nl/block-types/state%20ui/state_input_dynamic) | A typed input field (text, number, date, etc.). | `state` → form `content` | | `state_button` | The submit button with a click event. | `state` → form `content`; click triggers the submit flow | | [`state_form_get`](https://docs.rual.nl/block-types/state%20ui/state_form_get) | Reads the submitted form (`form-object` with all field values, `form-error` on failure). | first block of the submit flow | | [`mutations_set_bp_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_set_bp_field_multiple) | Maps form values to document fields. | `mutations` → create/update block | | [`function_create_document_from_mutations`](https://docs.rual.nl/block-types/storage/function_create_document_from_mutations) | Writes the document. | `flow` ← submit flow, `storage` ← `storage` block | Step-by-step with screenshots: [Build Your First CRUD App](https://docs.rual.nl/getting-started/first-crud-app). ## 3. Data Validation Reject bad input before it reaches storage or an external API. | Block | Purpose | Connects to | | --- | --- | --- | | [`condition_not_empty_value`](https://docs.rual.nl/block-types/condition/condition_not_empty_value) | `true` when the checked field is filled. | one per required field, fed from `form-object` or request params | | `branch` | Splits the flow on a condition into `true`/`false` paths. | `condition` ← the validation blocks | | [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json) (API) or a UI message (pages) | The rejection: 400 with an error object, or inline form feedback. | `false` path; `true` path continues to the save/work flow | Keep one condition block per field. The user learns exactly which field failed, instead of "something was wrong". ## 4. File Uploads Accept files through an API and store them as cluster assets. | Block | Purpose | Connects to | | --- | --- | --- | | [`on_startup_register_uri_post`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_post) | Receives the upload request (multipart body). | function handler | | Files-group blocks (search `file` in the block library) | Read the uploaded file, its name/type/size. | fed by the request body | | [`file_from_assets`](https://docs.rual.nl/block-types/files/file_from_assets) | Reads a file from cluster assets (`/public/...` or `/system_assets/...`). | `asset path` ← value; out `file`, `size`, `success` | | Asset upload API (`POST /api/v1/_system/assets/stream`) | Stores the file into `public` or `system_assets`. | alternative to a custom flow: often enough on its own | Everything about asset paths, public vs private, and the upload APIs: [Assets](https://docs.rual.nl/blueprints/assets). ## 5. Email Sending Send transactional email through Postmark, driven by a flow. | Block | Purpose | Connects to | | --- | --- | --- | | Any trigger (form submit, `storage_event`, [`schedule_repeating_event`](https://docs.rual.nl/block-types/events/schedule_repeating_event)) | Decides when an email goes out. | start of the flow | | Data blocks (search/get) | Collect recipient and content (user profile, order, template values). | value pins into the email fields | | Postmark send blocks (search `postmark` in the block library) | Send the message via the cluster's Postmark token. | to/subject/body from your data; token from `postmark_token` system setting | | [`function_console_log`](https://docs.rual.nl/block-types/logging/function_console_log) | Record success/failure while developing. | `message` ← send result or `error` pin | Prerequisite: `postmark_token` (secure) and `email_default_from` must be set. See [Email configuration](https://docs.rual.nl/blueprints/system-settings-reference#email-setup). For heavy or batched mail, defer to the [queue](https://docs.rual.nl/blueprints/queue) instead of sending inline. ## 6. Template or From Scratch? - **Use a template shape** when the job is a known pattern: endpoint, form write, validation, upload, notify. It saves wiring and keeps conventions uniform across your cluster. - **Go from scratch** when the data model or the flow is genuinely yours: unusual branching, multi-storage orchestration, custom UI state. Start from the closest template and prune rather than starting empty. - **Extract to a function** once you paste the same three blocks twice. See [Blueprint Best Practices](https://docs.rual.nl/blueprints/best-practices). ## Next Steps - [Common Blueprint Patterns](https://docs.rual.nl/blueprints/common-patterns): Recipes with notes and pitfalls: validation, auth, transformation, errors, caching. - [Browse All Blocks](https://docs.rual.nl/block-types): The full block library by group. - [Build Your First API](https://docs.rual.nl/cluster/api-quickstart): Apply the API-endpoint template end to end. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) --- --- title: "Finding Blocks · RUAL Documentation" description: "How to find the right block fast: search techniques, the group taxonomy, naming patterns, and starter sets per job." canonical: https://docs.rual.nl/block-types/finding-blocks language: en --- [Cluster](https://docs.rual.nl/cluster) [Blocks](https://docs.rual.nl/block-types) [Interfaces](https://docs.rual.nl/interfaces) [Blueprints](https://docs.rual.nl/blueprints) [Tutorials](https://docs.rual.nl/tutorials) [Home automation](https://docs.rual.nl/home-automation) [Examples](https://docs.rual.nl/examples) [Reference](https://docs.rual.nl/reference) [Architecture](https://docs.rual.nl/architecture) [Troubleshooting](https://docs.rual.nl/troubleshooting) Other # Finding Blocks 1,600+ blocks and growing. How to actually find the one you need: search techniques, the group taxonomy, naming patterns, and starter sets per job. The block library is huge on purpose. The trick is not knowing every block, but finding the right one in seconds. This page collects the discovery techniques that work. ## Search Techniques - **Canvas search**: right-click the canvas → Add new block, or press `Ctrl+F` in a blueprint, then type. It matches block names, display names, groups, and aliases. - **Search by verb**. Blocks are named by what they do: `search`, `create`, `update`, `remove`, `get`, `set`, `convert`, `send`, `render`. Start with the verb, not the noun. - **Search by system**: prefix with the thing you're touching: `storage`, `http`, `redis`, `postmark`, `state`, `query`, `array`, `object`, `function`. - **⌘K on this site**. The docs search (top-right) covers blocks live: same names, groups, and aliases, with links to each block's page. - **Block Search in Studio**. The cluster's Block Search view searches blocks across all your blueprints, handy for "where did I use that before?". ## The Group Taxonomy Every block belongs to a base group: browse them in [All blocks](https://docs.rual.nl/block-types). The groups you'll live in most: | Group | What's inside | Go there when | | --- | --- | --- | | `state ui` / `state elements` | Pages, forms, inputs, tables, layout | Building anything a user sees | | `storage` / `query` | Documents, search, filters, mutations | Reading or writing data | | `http connection` / `api` | Endpoints, requests, responses, auth | Building or calling APIs | | `array` / `object` | Map, filter, convert, fields | Transforming data between blocks | | `condition` / `flow` | Branches, loops, conditions | Controlling execution | | `function` | Triggers, returns, execution | Structuring reusable logic | | `redis cache` | Cache get/set/delete, TTL | Speeding up expensive flows | | Integrations (`postmark`, `discord`, `twilio`, …) | Third-party send/receive blocks | Talking to the outside world | ## Reading Block Names Type names are ` _ _ `: `function_search`, [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields), [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json), [`mutations_set_custom_field`](https://docs.rual.nl/block-types/mutations/mutations_set_custom_field). Once you read names that way, search becomes guess-and-check-free: need to update a document? Something with `update` and `document` → [`function_update_document_mutations`](https://docs.rual.nl/block-types/storage/function_update_document_mutations). Display names are the friendly version shown on canvas ("reply in json", "get form data"); type names are what you search and what block pages are titled with. ## Starter Sets by Job | Job | Blocks to know first | | --- | --- | | Build a page | `state_page`, [`state_render_page`](https://docs.rual.nl/block-types/state%20ui/state_render_page), `state_h1`/`state_form`/[`state_input_dynamic`](https://docs.rual.nl/block-types/state%20ui/state_input_dynamic), `value_default` | | Build an API | [`on_startup_register_uri_get`](https://docs.rual.nl/block-types/http%20connection/on_startup_register_uri_get)/`_post`, [`trigger_custom_function`](https://docs.rual.nl/block-types/globals%2Cfunction%20execution/trigger_custom_function), [`httpconnection_set_json`](https://docs.rual.nl/block-types/json%2Chttp%20connection/httpconnection_set_json), [`httpconnection_current_request`](https://docs.rual.nl/block-types/http%20connection/httpconnection_current_request) | | Read data | `function_search`, [`function_get_document`](https://docs.rual.nl/block-types/storage/function_get_document), `query_and`, [`query_bool_term_fields`](https://docs.rual.nl/block-types/query/query_bool_term_fields), [`query_sort_field`](https://docs.rual.nl/block-types/query/query_sort_field), `storage` | | Write data | [`function_create_document_from_mutations`](https://docs.rual.nl/block-types/storage/function_create_document_from_mutations), [`function_update_document_mutations`](https://docs.rual.nl/block-types/storage/function_update_document_mutations), [`mutations_set_bp_field_multiple`](https://docs.rual.nl/block-types/mutations/mutations_set_bp_field_multiple), [`mutations_set_custom_field`](https://docs.rual.nl/block-types/mutations/mutations_set_custom_field) | | Transform data | [`object_new_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_new_fields), [`object_update_fields`](https://docs.rual.nl/block-types/object%2Cfields/object_update_fields), `array_map`, [`array_convert_to_fielddefinitions`](https://docs.rual.nl/block-types/type%20casting/array_convert_to_fielddefinitions) | | Handle errors | success/error out-pins, [`function_console_log`](https://docs.rual.nl/block-types/logging/function_console_log), `branch`, [`condition_not_empty_value`](https://docs.rual.nl/block-types/condition/condition_not_empty_value) | | Background work | [`function_custom_execute_from_queue`](https://docs.rual.nl/block-types/function%20execution/function_custom_execute_from_queue), [`schedule_repeating_event`](https://docs.rual.nl/block-types/events/schedule_repeating_event), `storage_event` | ## Next Steps - [Block Templates](https://docs.rual.nl/block-types/block-templates): Proven combinations once you've found the parts. - [Browse All Blocks](https://docs.rual.nl/block-types): Every block by group, with pins and info. - [Tips & Tricks](https://docs.rual.nl/blueprints/tips-and-tricks): Canvas search, smart pins, and other discovery shortcuts. Was this page helpful? [Tell us what to improve](https://docs.rual.nl/support) · RUAL Docs is an integral component of the [RUAL ecosystem](https://rual.nl) ---