Support Package for Customizing Scenes

Vehicle Dynamics Blockset™ comes installed with prebuilt 3D scenes in which to simulate and visualize the vehicles modeled in Simulink®. These 3D scenes are visualized using the Unreal Engine® from Epic Games®. By using the Unreal® Editor, you can customize these scenes or simulate within the scenes from your own custom project.

With custom scenes, you can co-simulate within Simulink and the Unreal Editor so that you can modify your scenes between simulation runs. You can also package your scenes into an executable file and simulate without the Unreal Editor. Executable files do not require opening the Unreal Editor to simulate your scene. Instead, the scene runs by using the Unreal Engine that comes installed with Vehicle Dynamics Blockset.

To customize 3D scenes, you need to be familiar with creating and modifying scenes in the Unreal Editor. To simulate with these custom scenes, in your Simulink model, you must set the Simulation 3D Scene Configuration block parameter Scene source to Unreal Executable or Unreal Editor.

This table provides the steps for customizing the scenes.

Step

Description

Verify Software and Hardware Requirements (One-Time Step)

Verify the software and hardware requirements.

Install Support Package and Configure Scene Customization (One-Time Step)

Use the Add-On Explorer to install the support package that contains the Unreal Engine 4.23 project file.

Set up the 3D simulation environment to use the project file.

Migrate Projects Developed Using Prior Support Packages (Optional)

Migrate a project that you developed using a prior release of the Vehicle Dynamics Blockset Interface for Unreal Engine 4 Projects support package.

Open Unreal Editor from MATLAB

Establish a connection between the Unreal Editor, MATLAB®, and Simulink so that you can edit scenes.

Create or Modify Scenes in Unreal Editor

Use the Unreal Editor to customize the scenes in the project.

You can use the Simulation 3D Message Set and Simulation 3D Message Get blocks to send and receive scene data from the Unreal Engine 3D visualization environment. You must configure the visualization environment to communicate with the Simulink model data. For information, see the block documentation and the support package file QuickStart_Message_Get_Set.

Co-Simulate Scene in Unreal Editor and Simulink

Simulate the custom scene in both the Unreal Editor and Simulink.

Simulate Custom Scene Using Executable (Optional)

Create an Unreal Engine project executable file that contains your updates.

Verify Software and Hardware Requirements (One-Time Step)

Before creating custom scenes, make sure that your environment meets the minimum software and hardware requirements.

Software Requirements

  • Windows® 64-bit platform — The 3D simulation environment is not supported on Mac and Linux® platforms.

  • Unreal Editor 4.23 — To customize scenes, you must have this specific version of the Unreal Editor installed. To download this version, see the Unreal Engine website.

  • Visual Studio® 2017 or higher — This software is required for using the Unreal Editor to customize scenes.

  • Microsoft® DirectX® — If this software is not already installed on your machine and you try to simulate in the 3D environment, Vehicle Dynamics Blockset prompts you to install it. Once you install the software, you must restart the simulation.

  • Vehicle Dynamics Blockset Interface for Unreal Engine 4 Projects Support Package — For installation and setup instructions, see Install Support Package and Configure Scene Customization (One-Time Step).

Minimum Hardware Requirements

The 3D simulation environment also requires:

  • Graphics card (GPU) — Virtual reality-ready GPU with 8 GB of onboard RAM

  • Processor (CPU) — 2.60 GHz

  • Memory (RAM) — 12 GB

Install Support Package and Configure Scene Customization (One-Time Step)

Install Support Package

  1. On the MATLAB Home tab, in the Environment section, select Add-Ons > Get Add-Ons.

  2. In the Add-On Explorer window, search for the Vehicle Dynamics Blockset Interface for Unreal Engine 4 Projects support package. Click Install.

    Note

    You must have write permission for the installation folder.

Configure Scene Customization Using Support Package

The support package includes:

  • An Unreal Engine project, AutoVrtlEnv.uproject, and its associated files. The project includes editable versions of the prebuilt 3D scenes that you can select from the Scene description parameter of the Simulation 3D Scene Configuration block. To use this project, you must copy the file to a folder on your local machine.

  • A plugin file, MathWorkSimulation.uplugin. This plugin establishes the connection between MATLAB and the Unreal Editor and is required for co-simulation. You must copy this plugin to your local installation of the editor.

To copy the project to a local folder and the plugin to your Unreal Editor installation, follow these one-time steps. Use the Code That Configures Scene Configuration (Steps 1–4).

StepDescription
1

Specify the location of the support package project files and a local folder destination.

Note

You must have write permission for the local folder destination.

2

Specify the location of the Unreal Engine installation, for example C:\Program Files\Epic Games\UE_4.23.

3

Copy the MathWorksSimulation plugin to the Unreal Engine plugin folder.

4

Copy the support package folder that contains the AutoVrtlEnv.uproject files to the local folder destination.

Code That Configures Scene Configuration (Steps 1–4)

%% STEP1
% Specify the location of the support package project files and a local folder destination
% Note:  Only one path is supported. Select latest download path.
dest_root = "C:\Local";
src_root = fullfile(matlabshared.supportpkg.getSupportPackageRoot, ...
    "toolbox", "shared", "sim3dprojects", "automotive");

%% STEP2
% Specify the location of the Unreal Engine installation.
ueInstFolder = "C:\Program Files\Epic Games\UE_4.23";

%% STEP3
% Copy the MathWorksSimulation plugin to the Unreal Engine plugin folder.
mwPluginName = "MathWorksSimulation";
mwPluginFolder = fullfile(src_root, "PluginResources", "UE423"); % choose UE version
uePluginFolder = fullfile(ueInstFolder, "Engine", "Plugins");
uePluginDst = fullfile(uePluginFolder, "Marketplace", "MathWorks");

cd(uePluginFolder)
foundPlugins = dir("**/" + mwPluginName + ".uplugin");

if ~isempty(foundPlugins)
    numPlugins = size(foundPlugins, 1);
    msg2 = cell(1, numPlugins);
    pluginCell = struct2cell(foundPlugins);

    msg1 = "Plugin(s) already exist here:" + newline + newline;
    for n = 1:numPlugins
        msg2{n} = "    " + pluginCell{2,n} + newline;
    end
    msg3 = newline + "Please remove plugin folder(s) and try again.";
    msg  = msg1 + msg2 + msg3;
    warning(msg);
else
    copyfile(mwPluginFolder, uePluginDst);
    disp("Successfully copied MathWorksSimulation plugin to UE4 engine plugins!")
end

%% STEP4
% Copy the support package folder that contains the AutoVrtlEnv.uproject
% files to the local folder destination.
projFolderName = "AutoVrtlEnv";
projSrcFolder = fullfile(src_root, projFolderName);
projDstFolder = fullfile(dest_root, projFolderName);
if ~exist(projDstFolder, "dir")
    copyfile(projSrcFolder, projDstFolder);
end

Migrate Projects Developed Using Prior Support Packages

If your Simulink model uses an Unreal Engine executable or project developed using a prior release of the Vehicle Dynamics Blockset Interface for Unreal Engine 4 Projects support package, you must migrate the project to make it compatible with Unreal Editor 4.23. Follow these steps:

  1. Open Unreal Engine 4.23. For example, navigate to C:\Program Files\Epic Games\UE_4.23\Engine\Binaries\Win64 and open UE4Editor.exe.

  2. Use the Unreal Project Browser to open the project that you want to migrate.

  3. Follow the prompts to open a copy of the project. The editor creates a new project folder in the same location as the original, appended with 4.23. Close the editor.

  4. In a file explorer, remove the space in the migrated project folder name. For example, rename MyProject 4.23 to MyProject4.23.

  5. Use MATLAB to open the migrated project in Unreal Editor 4.23. For example, if you have a migrated project saved to the C:/Local folder, use this MATLAB code:

    path = fullfile('C:','Local','MyProject4.23','MyProject.uproject');
    editor = sim3d.Editor(path);
    open(editor);
    

    Note

    The R2020a support package includes changes in the implementation of some actors. Therefore, if the original project contains actors that are placed in the scene, some of them might not fully migrate to Unreal Editor 4.23. To check, examine the Output Log.

    The log might contain error messages. For more information, see the Unreal Engine 4 Documentation or contact MathWorks Technical Support.

  6. Optionally, after you migrate the project, you can use the project to create an Unreal Engine executable. See Package Scene into Executable Using Unreal Editor.

Open Unreal Editor from MATLAB

When editing custom scenes, if you open the project file directly in the Unreal Editor, MATLAB and Simulink do not establish a connection with the editor. Therefore, you cannot later simulate with these scenes in Simulink. Instead, you must you must open the editor from MATLAB by using the sim3d.Editor function. This function creates an Editor object that contains the path to your project file. You can then call the open function on this object to open the editor from MATLAB and start editing your project.

Note

The first time that you open the Unreal Editor from MATLAB, you might be asked to rebuild the UE4Editor DLL files. Click Yes to rebuild them.

Open AutoVrtlEnv Project Containing Prebuilt Scenes

To open the AutoVrtlEnv.uproject file that was copied from the Vehicle Dynamics Blockset Interface for Unreal Engine 4 Projects support package, specify the path to where you copied this project. For example, if you copied the AutoVrtlEnv.uproject to C:/Local/AutoVrtlEnv, as shown in the previous section, use this code:

path = fullfile('C:','Local','AutoVrtlEnv','AutoVrtlEnv.uproject');
editor = sim3d.Editor(path);
open(editor);

The editor opens the AutoVrtlEnv.uproject file. By default, the project displays the Straight Road scene.

Open Custom Project

To open a project that you developed using a prior release of the support package, see Migrate Projects Developed Using Prior Support Packages.

To open your own project, use the same commands used to open the AutoVrtlEnv.uproject file. Update the path variable with the path to your .uproject file. For example, if you have a project saved to the C:/Local folder, use this code:

path = fullfile('C:','Local','myProject','myProject.uproject');
editor = sim3d.Editor(path);
open(editor);

If this is your first time opening a custom project in Unreal Editor 4.23, you may need to associate, or reparent, this project with the Sim3dLevelScriptActor level blueprint used in Vehicle Dynamics Blockset. The level blueprint controls how objects interact once placed within in the Unreal environment. If you do not reparent the project to this level blueprint, the simulation returns an error.

To reparent the level blueprint:

  1. In the Unreal Editor toolbar above the editor window, select Blueprints > Open Level Blueprint.

  2. In the Level Blueprint window, select File > Reparent Blueprint.

  3. Click the Sim3dLevelScriptActor blueprint and close the Level Blueprint window.

    If you do not see this blueprint, make sure that you have the MathWorksSimulation plugin installed and enabled.

    1. Navigate back to the editor window. In the toolbar above this window, select Settings > Plugins.

    2. In the Plugins window, verify that the MathWorks Interface plugin appears. This plugin refers to the MathWorksSimulation.uplugin file that you copied into your local Unreal Editor installation from the Vehicle Dynamics Blockset Support Package for Unreal Engine 4 Projects.

      If you editor installation includes the plugin, then when you open a project in the editor for the first time, you are prompted to enable this plugin. If you do not see the MathWorks Interface plugin in this window, repeat the steps under and reopen the editor from MATLAB.

    3. Select the Enabled check box.

    4. Close the editor, reopen it from MATLAB, and repeat the steps to reparent the blueprint.

Create or Modify Scenes in Unreal Editor

After you open the editor from MATLAB, you can modify the scenes in your project before co-simulation. You can also create scenes in your project.

Open Scene

In the Unreal Editor, scenes within a project are referred to as levels, and the specific level type for these scenes are known as maps.

To open a prebuilt scene from the AutoVrtlEnv.uproject file, in the Content Browser pane below the editor window, navigate to the Content > Maps folder. Then, select the map that corresponds to the scene you want to modify.

Vehicle Dynamics Blockset Scene

Unreal Editor Map

Curved Road

HwCurve

Double Lane Change

DblLnChng

Open Surface

BlackLake

Large Parking Lot

LargeParkingLot

Parking Lot

SimpleLot

Straight Road

HwStrght

US City Block

USCityBlock

US Highway

USHighway

Note

The AutoVrtlEnv.uproject file does not include the Virtual Mcity scene.

To open a scene within your own project, in the Content Browser pane, navigate to the folder that contains your scenes.

Send Data to Scene

The Simulation 3D Message Get block retrieves data from the Unreal Engine 3D visualization environment. To use the block, you must configure scenes in the Unreal Engine environment to send data to the Simulink model.

For detailed information about using the block to customize the scenes, see the support package file QuickStart_Message_Get_Set and the Simulation 3D Message Get documentation.

Receive Data from Scene

The Simulation 3D Message Set block sends data to the Unreal Engine 3D visualization environment. To use the block, you must configure scenes in the Unreal Engine environment to receive data from the Simulink model.

For detailed information about using the block to customize the scenes, see the support package file QuickStart_Message_Get_Set and the Simulation 3D Message Set documentation.

Create New Scene

To create a scene in your project, from the top menu of the editor, select File > New Level.

Alternatively, you can create a scene from an existing one. This technique is useful, for example, if you want to use one of the prebuilt scenes in the AutoVtrlEnv project as a starting point for creating your own scene. To save a version of the currently opened scene to your project, from the top menu of the editor, select File > Save Current As. The new scene is saved to the same location as the existing scene.

Modify Scene

You can modify scenes in a project by dragging and dropping scene objects in the Unreal Editor. Alternatively, you can select a scene object in the editor window and then modify its properties in the Details pane on the right. For example, to change the position, orientation, or size of an object, in the Details pane, use the options in the Transform section.

The Unreal Editor uses a left-hand Z-up coordinate system, where the Y-axis points to the right. When positioning objects in a scene, keep this coordinate system difference in mind.

For more advanced information on modifying scenes in the Unreal Editor, see the Unreal Engine 4 Documentation.

Add Objects to Scene

To add objects to a scene, you can browse or search for objects in the Content Browser pane at the bottom and drag them into the editor window.

When adding objects to a scene in the AutoVrtlEnv project (or a scene in a project that was based off this project), you can choose from a library of driving-related objects. These objects, known as static meshes, begin with the prefix sm_. Search for these objects in the Content Browser pane at the bottom.

For example, to add a traffic cone to a scene in the AutoVrtlEnv project:

  1. In the Content Browser pane at the bottom of the editor, navigate to the Content folder.

  2. In the search bar, search for SM_Cone. Drag the cone from the Content Browser into the editing window. You can then change the position of the cone in the editing window or on the Details pane on the right, in the Transform section.

To migrate these objects, also known as assets, from a AutoVrtlEnv-based project into your own project file, see Migrating Assets in the Unreal Engine documentation.

Co-Simulate Scene in Unreal Editor and Simulink

After you open a custom scene in the Unreal Editor, you can simulate this scene in both the editor and Simulink. By using this co-simulation framework, you can add vehicles and sensors to a Simulink model, and then play back this simulation in your custom scene. This kind of simulation workflow enables you to tune your custom scene based on the simulation results before packaging the scene into an executable.

To set up Simulink co-simulation with the Unreal Editor, in your Simulink model, double-click the Simulation 3D Scene Configuration block. Set Scene source to Unreal Editor.

To run the simulation, in Simulink, click Run. Before you click Play in the Unreal Editor, wait until the Diagnostic Viewer window displays this confirmation message:

In the Simulation 3D Scene Configuration block, you set the scene source to 'Unreal Editor'.
In Unreal Editor, select 'Play' to view the scene.
This message confirms that Simulink has instantiated the scene actors, including the vehicles and cameras, in the Unreal Engine 3D environment. If you click Play before the Diagnostic Viewer window displays this confirmation message, Simulink might not instantiate the actors in the Unreal Editor.

By default, the Unreal Editor displays the scene from the scene origin. To display the scene from the perspective of a vehicle, in the Simulation 3D Scene Configuration block, select the vehicle name from the Scene view parameter. The next time you simulate within the editor, the scene displays from the vehicle origin, which is on the ground, beneath the geometric center of the vehicle. To change this view, use the numeric keypad in the editor.

KeyCamera View

1

Back left

2

Back

3

Back right

4

Left

5

Internal

6

Right

7

Front left

8

Front

9

Front right

0

Overhead

To restart a simulation, click Run in the Simulink model, wait until the Diagnostic Viewer displays the confirmation message, and then click Play in the editor.

If you are co-simulating a custom project, for the numeric keypad to work in the editor, copy the DefaultInput.ini file from the support package installation folder to your custom project folder. For example, copy DefaultInput.ini from:

C:\ProgramData\MATLAB\SupportPackages\R2020a\toolbox\shared\sim3dprojects\automotive\AutoVrtlEnv\Config
to:
C:\<yourproject>.project\Config

Simulate Custom Scene Using Executable (Optional)

When you are satisfied with your custom scene, you can package the project file containing this scene into an executable. You can then configure your model to simulate from this executable by using the Simulation 3D Scene Configuration block. Executable files can improve simulation performance and do not require opening the Unreal Editor to simulate your scene. Instead, the scene runs by using the Unreal Engine that comes installed with Vehicle Dynamics Blockset.

To simulate a scene by using an executable, first package the project that contains the scene into an executable. Then simulate this scene in your Simulink model.

Package Scene into Executable Using Unreal Editor

  1. Open the project that contains the scene in the Unreal Editor. Use the sim3d.Editor function. For more details, see the Open Unreal Editor from MATLAB section.

  2. In the menu above the editor window, select Settings > Project Settings to open the Project Settings window.

  3. In the left pane, in the Project section, click Packaging.

  4. In the Packaging section, set these options:

    Packaging OptionEnable or Disable?
    Use Pak FileEnable
    Cook everything in the project content directory (ignore list of maps below)Disable
    Cook only maps (this only affects cookall)Enable
    Create compressed cooked packagesEnable
    Exclude editor content while cookingEnable

    If you do not see all these options, at the bottom of the Packaging section, click the Show Advanced expander .

  5. Specify the scene from the project that you want to package into an executable.

    1. In the List of maps to include in a packaged build option, click the Adds Element button .

    2. Specify the path to the scene that you want to include in the executable. By default, the Unreal Editor saves maps to the /Game/Maps folder. Therefore, if the /Game/Maps folder has a scene named myScene that you want to include in the executable, enter /Game/Maps/myScene.

    3. Add or remove additional scenes as needed.

      Note

      Packaging a project into an executable can take several minutes. The more scenes that you include in the executable, the longer the packaging takes.

  6. Close the Project Settings window.

  7. In the top menu of the editor, select File > Package Project > Windows > Windows (64-bit). Select a local folder in which to save the executable, such as to the root of the project file. For example, C:/Local/myProject.

    Once packaging is complete, the folder where you saved the package contains a WindowsNoEditor folder that includes the executable file. This file has the same name as the project file.

    For example, suppose you package a scene that is from the myProject.uproject file and save the executable to the C:/Local/myProject folder. The editor creates a file named myProject.exe with this path:

    C:/Local/myProject/WindowsNoEditor/myProject.exe

    Note

    If you repackage a project into the same folder, the new executable folder overwrites the old one.

Simulate Scene from Executable in Simulink

To improve co-simulation performance, consider configuring the Simulation 3D Scene Configuration block to co-simulate with the project executable.

  1. In your Simulink model, open the Simulation 3D Scene Configuration block.

  2. Set these parameters:

    • Scene source to Unreal Executable.

    • File name to the name of the Unreal Engine executable file, specified as a valid executable name. You can either browse for the file or specify the full path to the file by using backslashes. For example:

      C:\Local\myProject\WindowsNoEditor\myProject.exe

    • Scene to the name of a scene from the within the executable file, specified as the path to a valid scene name. For example:

       /Game/Maps/myScene

  3. Run the simulation. The model simulates in the custom scene that you created.

Troubleshooting

Startup Warning

If you start Unreal Engine without following the steps in Open Unreal Editor from MATLAB and Co-Simulate Scene in Unreal Editor and Simulink, you can get a warning that reads: Warning: Integration with MATLAB/Simulink is not active. Unreal Engine might crash if you then continue working with the Unreal Editor.

Error Creating Project Executable

To create a project executable, the MathWorksSimulation plugin must be located in the Unreal Engine plugin folder. Check that the MathWorksSimulation plugin is not in the AutoVrtlEnv folder or subfolders. Set up the environment by following the steps in Open Unreal Editor from MATLAB.

See Also

| | | |

Related Examples

More About

External Websites