Experience League | Image

Create Immersive Digital Experiences for Retail Using 3D and AR

Table of Contents

Lesson 1: Introduction to 3D graphics

Basic concepts of 3D

3D graphics defines a visualization method that utilizes three-dimensional assets for creating many different types of content, from real-world recreations, to the abstract and everything in between.

This content is extremely versatile where it can be used for creating still images, animations, real-time interactions in games, and the applications are even expanding into the real world with technology like Augmented and Virtual Reality and 3D printing.

The fact that it is set in 3 dimensions, just like the real world, makes this workflow very intuitive and convenient.

3D graphics are being used in multiple industries:

  • Motion pictures (VFX)
  • Animated movies
  • Product engineering
  • Architectural visualization
  • Video gaming industry
  • Education
  • Future technologies, like VR, AR, 3D printing …

3D Scene

A 3D scene is a three-dimensional space which typically contains camera and 3D objects. Users are able to orbit and/or animate the camera in space and manipulate the objects in order to compose a scene. Each of the scene’s objects has XYZ space coordinates, rotation and scale values.

3D Object

3D objects are the primary medium used in a 3D scene. They are sometimes referred to as meshes. They are the representation of every and any thing from basic shapes to organic and abstract elements. They are made up of many vertices (dots in 3D space). Those dots are connected to make polygons, which ultimately make up the 3D object itself.

UV map

A UV map is an unwrapped and flattened 2D representation of a 3D object. It places each of the object’s vertices onto a two-dimensional surface. The horizontal axis of this quadrant is U, while the vertical axis is V (hence UV). UV maps are used to map 2D textures around a 3D surface. They are an embedded property that are stored with their respective 3D object.

A good way to visualize UV maps is by pretending the 3D object is made of paper, and you use a razorblade to carefully slice it into pieces that can be laid flat.

Textures

Textures are 2D images that are applied to a 3D object using its respective UV map. They are an important component of a material. They most commonly represent a 3D object's base color, but can also be used to customize other elements, such as reflection, roughness, or normal direction.

Materials and Shaders

A material is a combination of textures and additional variables. It ultimately defines how 3D objects in a scene look with its respective lighting. A material typically has a variety of properties which users can control (color, reflection, roughness, etc). These properties are controlled by sliders, colors or textures.

Materials are generally governed by a shader, which is code that provides additional customizations on how a 3D object looks. By changing the shader in a material, you can make something look cartoony, realistic, or abstract without altering the 3D object or texture.

Lights

Lights interact with the materials applied to 3D objects in the scene and allow them to be visually represented through a rendering process.

They simulate the way light photons are being transported in real life using a process called raytracing.

Rendering

3D rendering typically refers to process of creating a 2D image by processing the interaction of lights and scene’s objects which have materials associated to them. It is computed by software and highly dependant upon the render settings. These settings will determine the quality but also the render time.

Another type of rendering is real-time rendering, where 2D frames are calculated fast enough for real-time interaction as seen in game engines like Unity and Unreal.

Adobe Dimension, UI and basic functions

Dimension has been designed with designers and other non-3D experts in mind. As such, it aims to take the complexity away, while still delivering high end results. It is easy to learn and is relatable to existing Adobe users as it share the same UI paradigm as other Adobe’s tools, like Photoshop, Illustrator, etc.

Key features:

  • Scene assembly using free, custom made or Adobe Stock 3D models
  • Material assignment, texture editing with Photoshop option
  • Lighting setup
  • Seamless background compositing
  • Photorealistic rendering

At the moment, Dimension is not capable of geometry creation or direct texture editing/layering.

In this lesson we will cover the UI and some of the basic features of Dimension. Dimension has two separate modes: Design mode, where the scene is being assembled and Render mode, where the final image is being processed (or rendered)

This is the Design mode interface, where we’ll spend most time (Render mode will be covered later):

Online tutorials: https://helpx.adobe.com/dimension/tutorials.html

Scene Panel

Scene panel contains all the elements in the scene:

  • Environment: The lights and the background image are stored here as well as the ground plane properties
  • Camera: Camera properties (coordinates, focus and field of view)
  • Objects: You can see all the objects in the scene here; you are also able to group objects together. You can access their materials by double clicking on them.

Tools Panel

  • Select and Move Tool (V): Move objects in the scene
  • Select and Scale Tool (S): Scale objects in the scene
  • Select and Rotate Tool (R): Resize objects in the scene
  • Magic Wand Tool: Select certain areas of an object
  • Sampler Tool: Sample a material from one object to another
  • Orbit Tool (1): Orbit the camera; you can also use Right Mouse Button
  • Pan Tool (2): Pan the camera; you can also use Middle Mouse Button
  • Dolly Tool (3): Zoom the camera; you can also use Mouse Scroll
  • Horizon Tool: Move the horizon up and down
  • Zoom Tool: Zoom the canvas
  • Hand Tool: Pan the canvas

Object Properties

Here, you are able to adjust object’s position and rotation in space as well as its scale by entering numeric values. Also, you can change the position of its pivot.

Libraries

There are 4 types of libraries in Dimension, which contain ready-to-use free assets:

  • Models
  • Materials
  • Lights
  • Images

Adobe Stock (stock.adobe.com/3d-assets) contains a huge selection of both, free and paid assets from each of the categories listed above.

Lesson 2: 3D for Prototyping

3D graphics can be used for testing various apparel design variations. Dimension has a specifically designed set of tools, which make this iterative workflow smooth and efficient.

The scene in this exercise features a mannequin wearing a shirt, which needs to be designed. Therefore, the shirt’s material is the only one we need to worry about. Our goal is to try different colorways, edit them and see how we can apply a logo, using pre-made Illustrator templates.

The scene already has two camera bookmarks, allowing you to quickly swap between the front and the rear view.

The final results will be rendered out as 2D images which can be added into AEM.

Enable AEM Desktop Actions

  1. Open your browser to http://localhost:4502 and login as admin/admin

  2. Click on the user icon on top right of the screen, then click My Preferences

    50

  3. Enable Show desktop actions for Assets and click Accept.

    19

Login to AEM Desktop app

  1. Select preferences from the AEM Desktop App icon from the menu bar

    83

  2. The AEM Assets URL should read http://localhots:4502

    94

  3. Click Save & Close to keep any changes.

  4. Select Connect from the AEM Desktop icon in the menu bar

    18

  5. Enter admin/admin for the login and click Sign In

    16

  6. You should see the message Successful connection; Successfully connected to AEM. Click here to learn more.

    67

  7. With a successful connection you should be able to browse AEM folders on your file system.

    39

Browse AEM & Open Dimension file

  1. Switch to the chrome browser with AEM window open

  2. Click on the AEM logo on top left and select Assets > Files

    10

  3. Navigate to > Assets > Summit 2019 > 3 Dimension > 3-1 Mannequin

  4. Select the Checkmark as you hover over prototyping_apparel_START.dn (You can ask the TA how to maintain a version of this)

    92

  5. From the top toolbar, select Desktop Actions > Open (you can check "Always..." and then "Open...")

    41

  6. The file prototyping_apparel_START.dn open in Dimensions CC

    78

Apply Base Color using Illustrator Artwork

  1. Select the Outside Fabric object, located in the Scene panel > Standing Mannequin > Shirt folder (you can also CTRL+click on the shirt).

    13

  2. Select the shirt’s material (it is called “Shirt Material”) by double clicking on the object in the Scene panel.

    69

  3. We are only going to tweak the Base Color parameter. Click on its color/texture input, switch the mode from Color to Image

    72

  4. Load the colorways.ai file.

    1. Make sure your “AEM Desktop” is connected

    2. Open your Finder window to > localhost > DAM > summit-2019 > 2-colorways > 2-2-uv-maps

    3. Drag the colorways.ai file to dotted box in Dn.

      47

  5. Dimension is able to read the Artboards if they are included in the Ai file. Therefore, when you load the colorways.ai file, you will be able to pick which artboard you want to use as a color texture.

    95 89 46

  6. You are also able to edit the selected artboard, by clicking on the pen icon. This will take you to Illustrator, where you can make changes to the file (on one or more artboards).

    55

  7. Try changing the existing colorways and save the file:

    1. Switch to Direct Selection Tool
    2. Click on the desired UV map
    3. Click Fill to open color picker 35

    The changes will be reflected in Dimension as soon as the file is saved:

  8. You can save your colorways by saving the Illustrator file into > localhost > DAM > summit-2019 > colorways > uv-maps with the name colorways.ai (click on the filename to save time). Note there is a 30 second delay before upload to AEM. You can check upload status in AEM Desktop app > View Asset Status.

    62

Apply Logos

  1. Now, we’ll try adding the logo. Drag and drop logos.ai from localhost > DAM > summit-2019 > colorways > logos directly onto the shirt object in the Dimension scene viewport. The logo (called “Graphic”) will appear right above the material in the Scene panel hierarchy. You can change its size, position and rotation using the gizmo. Furthermore, you can tweak its material properties like Roughness and Metallic values. Try centering the logo on the shirt.

  2. Once again, you are able to pick the Artboards if they are included in the loaded Graphic’s file. Try swapping the logo by changing the artboards.

    9

  3. Click on the edit button of the Graphic, so we can change it in Illustrator, just like we did with the colorways file. Try changing the color:

    44

  4. Finally, let’s add a logo on the back. Change the camera bookmark from front to back, so we can see the back of the shirt.

    97

  5. This time, we’ll use an existing graphics from the library. Go to the Images library, drag and drop “Brand logo” file onto the shirt object and position it where you like.

    21

Render your Design

  1. It’s time to render out our design so it can be added to AEM. You can set the resolution by clicking on its current value in the upper left corner of the canvas. Set the resolution to 1000x1000px (smaller resolution will render faster).

    100

  2. Change the mode from Design to Render.

    42

  3. Here, you can select which view you would like to render. Pick front and back views and select High quality render. Finally, set the path (localhost > DAM > summit-2019 > 4-renders) and hit render. Dimension will now process the scene elements (lights, materials and objects) and produce 2D images (renders). However, you are also able to save the renders in the current state, even before the rendering has been fully completed. This feature is called Take a Snapshot and we’ll use it after letting Dimension render for about a minute or so. This will save us some time.

    12

Publishing 3D Scene to Web

  1. Dimension is capable of publishing the entire 3D scene to the Web, enabling users a more immersive experience, where they are able to rotate, move and pan the camera. They can also leave comments, so it’s a convenient way of interactive collaboration in the early stages of prototyping the design.

    7 2 32 6

Lesson 3: 3D for Visualization

In this exercise, we’ll use Dimension to create an assortment deck, used for merchandising purposes. The goal is to have a visual overview in InDesign, so it is easy to pick and drop desired colorways.

Typically, this process is done using actual photographs of samples, or, in many cases - a simple 2D sketch is used. Having a 3D render is a much more informative solution.

We’ll use the same workflow as in the previous exercise, but this time the focus is to create a spread of designs, rather than tweaking the existing ones.

  1. Open the scene visualization_apparel_START.dn from DAM/Assets > Summit 2019 > Dimension > Assortment via browser or Finder (Save or Discard changes to file already open in Dimension)

    74

  2. The scene contains 4 identical shirts placed on their mannequins. We also have 5 pre-saved camera bookmarks - one for each shirt and one that shows all of them.

    87

  3. Select the shirts one by one, go to their materials and load the file localhost > DAM > summit-2019 > 2-colorways > 2-2-uv-maps > colorways.ai into their Base Color channels.

    64 63

  4. Now, all 4 shirts should have the same colorway. Since, the Ai file we’ve used has 4 different colorways, we can go ahead and pick a different artboard/colorway for each shirt. This way, we’ll create a spread of design variations.

  5. We’ll do the same thing with logos: drag and drop the file logos.ai to all the shirts and then select and position a different artboard/logo for each one.

  6. The final step is to render out each colorway, so it can be used in InDesign. Change the canvas size to 250 x 250

    68

  7. Go to the Render tab and select colorway 01 - 04. If you want to have the entire deck on the same render you may also select “all colorways”. Select High quality settings, export format as PNG, set the output path to localhost > DAM > summit-2019 > 5-assortment and start the rendering process. Dimension will render out individual colorway images.

    29 5

  8. Once the png’s are rendered, navigate to DAM/Assets > Summit 2019 > Assortment via browser or Finder and open the file Visual Assortment_START.indd

    56

  9. Have your rendered png’s open in finder so you can drag and drop your visuals onto the assortment deck

  10. Optionally, you can also drag color chips from the DAM/Assets > Summit 2019 > 2 Colorways > 2-4 Color chips folder

  11. You can also drag color chips from the DAM/Assets > Summit 2019 > Colorways > Color chips folder

Lesson 4: 3D for Creative Marketing

Dimension can be used to produce both photo-realistic and highly stylized renderings, by composing the 3D elements with a background photo.

Once rendered, the image can be further edited in Photoshop. In this exercise, we’ll create an image which could be used for (fictional) marketing purposes.

  1. Navigate to AEM > Assets > Summit 2019 > 3 Dimension > 3-3 Posed, open the scene creative_marketing_apparel_START.dn (save or discard changes to dn file already open in Dimensions)

    28

  2. First, let’s plug the colorway and the logo to the shirt, just like we did before (or change the existing one in creative_marketing_apparel_START.dn)

  3. Then, we’ll add some materials to other objects in the scene:

    1. Select the Stool object folder in the Scene panel and assign the “Metal” material from the Material library.

      33 77

    2. Now select just the seat (CTRL+click on the object) and assign the “American Cherry Wood” material; in the material properties, set Repeat to 2 and Rotate by 90 deg.

      51

    3. Select the Pants object, assign the “Denim” material and set Repeat to 3

    4. Select the Mannequin object and then use the Sampler Tool (I) to pick and assign the metal material from the chair.

      17

  4. Now, let’s add the background: go to the Image library and drag and drop the “Gallery” image into the empty space of the Scene’s viewport (note: if you drag it on top of an object, you will create a Graphic instead). Alternatively, you can use the background_abstract.jpg or background_beach.jpg images, located in the project folder. Simply drag them from the file browser into an empty space of the Scene’s viewport.

  5. It is clear that the background image doesn’t quite match the 3D scene. To fix this, we can use the Match Image function, found in the Actions, with Environment selected. Match Image will try to perform a set of actions, using the machine learning algorithm:

    1. Resize canvas to match the image size ratio or the image size (in out case, we can deselect this option)

    2. Create Environment Image (makes an IBL out of the background image)

    3. Match Sunlight (if possible)

    4. Match Camera Perspective (if possible)

      40

  6. Now, the scene looks integrated into the background more closely (which can be seen by observing how materials like metals are reflecting the newly created environment and how the perspective matches the background). However, we can fine adjust the lights, reposition the sunlight, increase the intensity etc. to match it even better with the environment. You can also adjust the Ground plane properties: its roughness and its opacity.

  7. We are ready to render out the scene, just like in the previous example. Set the canvas size to 250 x 250, switch to the Render mode, pick High quality settings, select the output path summit-2019 > 6-creative > 6-2-landing-page-banners > completed and render out the image.

  8. When the render is complete you will get a layered PSD document. The document contains the background and the 3D elements on separate layers, so you can independently edit the two. Furthermore, you’ll get a several “utility” layers:

    • Material selection: a layer where each material has a plain color assign to it. This makes it easy to select and edit the material in Photoshop.

    • Object selection has a similar logic as the material selection, but here, colors are assigned to each independent object in the scene.

    • Depth pass is a grayscale, gradient layer, whose values are tied to the proximity of the 3D objects to the camera; the closer an object gets to the camera, the brighter it’s going to be and vice-versa. This layer can be used to emulate depth of field, using the Lens Blur filter in Photoshop. It can be also used as environment fog if it’s inverted and has the screen blending mode applied.

  9. The power of 3D allow you to create photorealistic images and go even beyond that, where you are able to pursue any artistic direction you might have. Here are a several variations, which were created by changing just a few parameters shown above with some light adjustments in Photoshop:

Lesson 5: AEM and DN Cloud Rendering

AEM does not have native 3D rendering capabilities. Rendering a 3D scene is computationally intensive and for some renders, requires hours to complete when rendering locally. DI3D Cloud Rendering Service reduces the time to render and offloads render processing to remote servers. In this exercise, we use the AEM and DI3D cloud rendering services integration to create thumbnails and renditions of Dimension CC files uploaded to AEM. This integration will enable AEM to render Dimension scenes (.dn files) directly from AEM Assets

  1. Open Dimension CC and create a new Dn file (File > New)

    71

  2. Create a simple 3D scene by dropping a coffee model from Models onto Canvas

    53

  3. Save the file to your desktop, name it <SEAT> – <LAST_NAME>.dn. For example, for a student named Bond who is sitting in seat 007, they would name their file 007 – Bond.dn

  4. Switch to the chrome browser with AEM window open

  5. Click on the AEM logo on top left and select Assets > Files

    10

  6. Navigate to Assets > Summit 2019 > 3 Dimension

  7. Upload the dimension file created above to AEM (you can also save the file to desktop app and sync it to AEM)

    49 84

  8. You can see the banner 3D Rendition unavailable on file, as AEM does not have native 3D rendering capabilities

    58

  9. Select the file and click DI3D Convert in toolbar

    37 4

  10. AEM authenticates with the Dimension cloud rendering service using IMS and submits a generate renditions request of selected file. The process may take few minutes for completion. AEM UI shows the processing states... (refresh screen)

    20

  11. After completion, renditions are downloaded and added in AEM for thumbnail views

    73

Lesson 6 (optional): Adding Dn Rendered Assets in AEM Sites

  1. Next we will use our photorealistic creative assets for a landing page banner on our online store

  2. Click on the Adobe Experience Manager banner at the top

  3. Select Sites

  4. Then navigate to we.retail > United States > English > Experience

  5. Select one of the pages there with a check mark

  6. Then click Edit from the tool bar

  7. First let’s open our side panel

  8. We can now see our creative assets

  9. Click on the Hero Image and make sure that web component is unlinked

  10. Now click the wrench icon to edit the web component

  11. This brings up the Image edit popup

  12. Drag your image over the existing one

  13. And click the checkbox to save

  14. You can click Preview to see your page without editing controls

  15. Next, let’s give our customer an immersive augmented reality experience. A usdz file has been pre-generated for you

  16. While in edit mode, click on an empty component

  17. And click the plus icon

  18. Select a Text type component then click the pencil to edit

  19. Type in some text and then select what you want to hyperlink and then click the link button

  20. Click the checkbox next to the Path field to navigate to the usdz file

  21. Navigate to Assets > Summit 2019 > Creative > AR usdz and select (checkmark) the .usdz file there and click Select

  22. The path should be filled for you, so check the Blue check box to save the path

  23. Then click the checkbox at the end to save the text component

  24. Next go into Preview mode to test the link. You can access the page and AR model with an iPhone at http://enterprise-demo.com:4502/content/we-retail/us/en/experience/wester-australia-by-camper-van.html