Deborah R. Fowler
USD and Houdini
Posted Dec 2 2019
Updated on Nov 5 2024
also on https://youtu.be/jmOyAcHTFC0
See also command line rendering in karma
Example file to create sequences of USD files (for example from simulation) use a USD clip node (Thanks Davis for the tip)
With Houdini 18 came the introduction of USD or Universal Scene Description developed by Pixar. It is necessary to also introduce a few other items:
- Solaris - Houdini's tool to create and edit USD. A network context called LOPs for creating layout, lighting, rendering.
- Karma (see also Houdini and Karma),
the Viewport becomes your renderview (Hydra-based IPR), and you
use a karma lop (node) with "save to disk" to render your scene
inside a lop network context (using USD).
- TIP you must create your render folder
- TIP variance threshold (limits the samples, pixel samples is max). Type d to change the preview settings.
- http://www.sidefx.com/docs/houdini/nodes/lop/karma.html
- cpu and xpu options in Karma and will replace mantra
- LOPs - to be able to set up your scene in USD format to
use Karma, you will use a lops network. It is a way to organize
your entire scene.
documentation tutorials AND FILES on LOPS/USD (more and more are being produced)
- highly recommend Matt Estella's pages
- CG Forge Karma & LOPs https://www.youtube.com/watch?v=y2ywi_c-ftg
- small note, rather than merge, could just add the nodes in
sequence
- CG Forge Karma & LOPs part II - lightmixer, light linking, render settings https://www.youtube.com/watch?v=wLJmr5d78I0
- TIP - to move your camera, lock and move
- TIP - once geometry is in LOPs, it is referred to as a "prim" or primitive, not the same as a Houdini primitive
- TIP - you can view your USD scene with a network panel USD/Scene Graph Tree
- Scene Graph Tree which gives a tree view of your USD scene,
editable (not the same as a SOP network tree view)
Now that we are in 20.5, USDs will be the underlying file format that is gaining popularity.
Sop import node note for 20.5
One change in 20.5 to the Sop import node from previous versions is that it now issues a warning if you have not specified an output path.
USD - no, not US dollar, but "Universal Scene Description" developed by Pixar.
USD is a toolset for reading, writing, editing and previewing 3D data. It is "stackable" in non-destructive layers.
Hydra is an API for generating an image from a USD scene. USD consists of C++ libraries with Python bindings for scripting.
- Pixar documentation
- NOTE: ignore Pixar USD documentation section A-Tour-of-USD-Houdini-Primitives.html
- at the time of this posting this section is out of date
- usdview will show you the contents of a usd file (you
can run the command from the command line rendering tool (from
houdini launcher)
- Houdini implementation
- You can create an ASCII readable version from your USD ROP by
adding an a to the extention ie. .usda
- to convert USD files from binary to ASCII use usdcat
- From the Houdini Command Line Tools 18.0.287: usdcat file.usd --out file.usd --usdFormat usda (NOTE -- is a double dash)
- usda is the extenstion for ascii, usdc is binary
- Example files of writing, converting and reading USD here
- you can also view a LOP node by RMB LOP
Actions->Inspect->Flattened Stage
- command line rendering
- SIGGRAPH Asia 2019 (posted Dec 3, 2019) presentation by Luke Gravett - Animal Logic using Solaris/USD
- vex and usd https://www.sidefx.com/docs/houdini/solaris/vex.html
Advantages:
- universal/standardization - being able to use the files across software
- USD have a "Purpose" - so it is easier to use proxy geometry
- instancing can be used inside lops using addpointinstance
- override elements of assets using point attritubes (like style spreadsheets but better)
Overview and Terminology
NOTE: there is an introduction and overview to USD in
the sidefx docs, below is a very brief summary
Understanding USD will help you understand Solaris, but the
details are handled by the lops nodes.
USD has been used by Pixar and is now open source with the
intention of having a common file format. It consists of layers,
and sometimes an anology to photoshop is used in terms of layers.
In Houdini there is a prune node that "hides" layers. Working in
"stage" in Houdini is the resulting composition of your layers.
Stage is a USD term to compose layers.
- layers - allow artists in different departments to work on the same entity, great for collaboration
- provide non-destructive editing
- variants - allow assets to package up similar but varied assets in one so they can be easily selected. There is also a concept of inheritance.
When composing layers, opinions are considered, so a
property will be overridden if the layer has a stronger opinion.
Layers contains Prims (not like a Houdini prim but a USD
concept). Prims and their contents are organized into a layer.
It allows differing departments (modeling, shading, animation,
lighting, fx, rendering) to have their own format, but still be
readable and interchangeable.
In Houdini, regarding writing USD files, remember that USD stores
data, not nodes (like a jpg versus a psd file, or like a bgeo.sc).
Assets also have the concept of purpose - render or proxy
hydra is the API that talks to USD
The physics option really has a DOP running in the background, so
practice good complex scene practices when using it.
On the edit lop right click and select use physics.
Best practices - look at the multi-shot workflow for the market
scene.
Glossary of common USD terms (there are more than listed here)
- prim (primitives) - unit of USD, have a name, properties, children. Prims are nodes in the tree of objects that make up a stage.
- properties - are either attributes or relationships, attributes being more common, ie, size
- primvar - is a specific type of attribute - a namespace is added to influence how the Hydra api treats the value and is involved in instancing (like geometry attributes)
- metadata - named values attached to properties that change how they work
- layers - a .usd file is a layer (usdc or usda) but a usdz file can contain multiple layers in one file
- stage - the results of loading USD files and composing the layers
- sublayers - import contents over the current tree
- references - import contents by attaching to the existing tree
- variants - used for family of geometry, level of
detail, material assignments (seen in sidefx lops
tutorial 1)
Common nodes in Houdini related to USD/Solaris
- sopcreate - is just a sop network
- layerbreak - then usd file only saves from there if it were to be written, creates a new layer manually
- sopimport - allows you to reference geometry from your sop context (creates an in-memory USD representation for use in Solaris)
- prune - hides/makes invisible a layer
- merge - in Solaris has options for how the layers are merges (sublayer, merge)
- editlop - allows you to use the physics option by RMB and select use physics
- configure nodes - allow you to change the "kind" of the asset using a configure layer, primitive lops
- stagemanager - SideFX recommends using a stagemanager node (seen in sidefx lops tutorial 2)
Example files
In the documentation
noted above, there is a "quick example", however I was not
able to use the two_boxes part of it verbatim (perhaps there is a
way, however I found a solution that works). Here is what I found
works with usda files and hipnc supplied below.
box.usda works great! Here are my versions:
#usda 1.0
def Cube "box"
{
double size = 0.5
}
This "def" defines a new prim called box. The "attributes" now
called properties are defined in the curly brace. Coming from a
C++ background, I tend to match my curly braces with each on its
own line (personal preference).
The two box example as listed in the docs as of this posting I
could not get to read in properly. The syntax I used was
#usda 1.0
def "box1" ( references = @./box.usda@</box> )
{
}
def "box2" ( references = @./box.usda@</box> )
{
double size = 5.0
}
Click
here for zip file of usda files and hip bringing them in
This defines the two boxes by referencing the other usda file
where box is defined as a primitive. If you look at the hip file
provided with the usda files you will see this can be imported
into Solaris with a file node or alternatively into Obj/geo
context with a USD import node. Both render perfectly in Karma
(ignore the odd looking corners in the Vulcan viewport).
This is a good start to understanding the layering of USD file
format. We will delve deeper as we talk about command line
rendering. Note that you are more likely to be using usd (or usdc)
rather than ascii readable file as binary will be smaller. In
addition, you will be generating the usd source from the interface
with a usd rop rather than these that were created in notepad++.
A few key points regarding Sops vs Lops
- You can work in a lopnetwork or in Solaris
- Scene Graph Tree allows manipulation - it is not like the "Tree View" of a sop network (shift w in network view)
- Sops modify geometry vs Lops which modify hierarchy and objects
- Chaining sopimports together consecutively in Lops appends
results - Sops chains are about geometry modification - in /obj,
parents
- In Lops, defaults are set up to help organize, such as all lights under a light tab, cameras, materials, etc.
- Walking the node network in both is convenient
- Write out your files using a usd rop (same as you will for the renderfarm) and then use a topnet in /obj with a filepattern and usdrender (shift-v to start)
- Note - if you attempt to use husk on this file, it will not automatically create the render folder but the topnet will
- husk -f 1 -n 24 nameOfFile.usd
- (note it works fine but at the time of this posting I'm
still investigating resolving the token
productName.timeSamples with abs paths)
- the above would do the equivalent of the topnet node from
the Houdini "command line tools" (access from launcher)
- Karma node in the regular /out context is actually a usd rop with a lopnetwork
- You can work in Sops and have a Lop network, or you can work in Lops(Solaris) and use a sopcreate which really is a sop network