Synthesis AI’s Face API: Inputs

Overview

The Face API is a data generating service that can create millions of labeled images of faces with a specifiable variety of genders, ethnicities, ages, expressions, eye gazes, hair, hats, glasses, and more.

A request (also known as a “job”) is typically submitted by using a command line client application provided by Synthesis AI, and a Job ID is returned. This job ID will then show up in the web interface, and is used as the reference for downloading images via the CLI. For those who specify an API when the callback is returned, notifying the user that the job and all of its tasks have completed.

Job requests are described in JSON format, and a single JSON can describe many scenes (aka tasks).

Workflow

The standard workflow for the Synthesis AI Face API is:

  1. Register Account, Download & Setup CLI
  2. Select Identities for the Face API with the Identities API
  3. Create Face API job with input JSON
  4. Download job output with CLI
  5. Parse outputs into your ML pipeline

Important Concepts


Percent Groups per Parameter

Almost all parameters have the concept of percent, and then an array of settings for the parameter, which is the percent of scenes/images these should apply to.

Example:
You might want mouth open & eyes closed to be applied to 70% of images, several other expressions to be applied to 20% of images, and then the remaining 10% of images having neutral.

Note: The percentage allocated across all groups in each section must add up to 100% — an error will be returned if they do not.

In order to accomplish this, 2 percent array elements would be specified, the first for 70% with certain expression settings, and the second for 20% of images with different expression settings. The remaining 10% of images would be implicitly the neutral expression, because that parameter would not be set on them.

Parameters with percent groups are:

  • Expressions
  • (Head) Hair
  • Facial Hair
  • Eye Gaze
  • Head Turn
  • Headwear
  • Masks
  • Glasses

Example JSON of a request with percent groups follows. In this example, 100 renders will be produced, with 40% of them containing 2 types of face masks, and 60% of them containing no face masks:

40% Have Masks, 60% No Masks

{
  "render": {
    "resolution": {
      "w": 512,
      "h": 512
    }
  },
  "faces": [
    {
      "identities": {
        "ids": [80],
        "renders_per_identity": 100
      },
      "accessories": {
        "masks": [{
          "style": ["Duckbill", "DuckbillFlat"],
          "percent": 40
        },{
          "style": ["none"],
          "percent": 60
        }]
      }
    }
  ]
}


Value lists vs. Min/Max Ranges

For parameters with numerical settings, there are two types of collections of values that can be specified: the list, and the range/interval.

Lists are a discrete list of values (repeating allowed) that are randomly selected from for each scene, like [10, 20, 23]. They are not randomly “shuffled”, but rather randomly selected, so duplicates can occur.

To further control distribution in a list, values can be repeated in the proportion they are desired. For example, a list with values [3, 3, 3, 3, 4, 4] would have a two-to-one ratio (on average), of scenes with the value of 3 rather than 4 for that parameter.

On the other hand, ranges are set between two numbers, with a min and a max value specified. Any valid random number between these two, and including these two values will be selected (inclusive of min/max values). The distribution between the two values comes out to usually be a uniform distribution, but duplicates can occur.

Within a single percent array item, only a list or a min/max range can be used for a given parameter. However, two separate percent array items may use different choices for lists vs. ranges.

Example of a JSON request with value lists and ranges for a parameter in 2 different percent arrays follows. In this example, the intensity of the expressions for 20% of the renders will be (randomly) evenly distributed between 0 and 1, and in 80% of the renders, the intensity will be at .25 and .5, and twice as likely still to be .75 (because it’s repeated twice).

Value lists vs. Min/Max Ranges JSON Example

{
  "render": {
    "resolution": {
      "w": 512,
      "h": 512
    }
  },
  "faces": [
    {
      "identities": {
        "ids": [80],
        "renders_per_identity": 100
      },
      "facial_attributes": {
        "expression": [{
          "name": ["all"],
          "intensity": { "type": "range", "values": {"min": 0.0, "max": 1.0} },
          "percent": 20
        },{
          "name": ["all"],
          "intensity": { "type": "list", "values": [0.25, 0.5, 0.75, 0.75]},
          "percent": 80
        }]
      }
    }
  ]
}


Top Level Entries

The JSON request file has two primary parts, a render section, and a faces array section. The render section describes the image size and quality for all renders, and the faces array is a collection of scene parameter groupings. These parameter groupings describe how to adjust the subjects in the renders, as well as the camera and environment.

Full, minimal example of a valid request:

{
  "render": {
    "resolution": {
      "w": 512,
      "h": 512
    }
  },
  "faces": [
    {
      "identities": {
        "ids": [80]
      }
    }
  ]
}


Render

The render key/values are a single set of values applied to every scene in the job request.


Resolution

Specify width & height of the rendered images.

  • Default: 512x512
  • Min: 64
  • Max: 4096

Quality

Specify the supported render engines, noise threshold, and denoising (smoothing) functionality. A lower noise threshold is higher quality and takes longer to render.

  • Default: 0.017
  • Min: 0.001
  • Max: 1.0

Note: At the moment, only the vray render engine is supported.

Render Section Example

"render": {
  "resolution": {
    "w": 512,
    "h": 512
  },
  "noise_threshold": 0.025,
  "denoise": true
}


Faces Array

The faces array object is a list of sets of scenes to render, each element of the array with a particular specification as documented in the following sections.

A minimal "multiple scene-set" JSON

"faces": [
  {
    "identities": {
      "ids": [80],
      "renders_per_identity": 1
    }
  },
  {
    "identities": {
      "ids": [79],
      "renders_per_identity": 1
    }
  }
]


Identities & Renders Per Identity

Our facial identities each have a numerical ID (e.g. 79, 80), and are consistent between every run. Each ID also has consistent demographic information with it that is returned on output (gender, ethnicity, age). See more in output documentation.

NOTE: IDs are not contiguous. Some IDs do not exist and will yield a job error if requested.

To get a list of IDs that match your demographic requirements, our command line tool provides a faces ids subcommand (read more) to query for ids matching a set of selection parameters.

One or more IDs can be sent in per scene specification (per faces array element). The total number of images generated for the array element will be, in pseudocode:

length(faces.identities.ids) x faces.identities.renders_per_identity.

Identities id Array List

  • Default: no default, required
  • Note: maximum length of 10,000 values

Identities renders_per_identity Value

  • Default: 1
  • Min: 1
  • Max: 1000

Visual examples of varying identities (see also: full input JSON to generate them):


Facial Attributes

The facial attributes section describes parts of the head anatomy. This includes:

  • Expressions
  • Eye Gaze
  • Head Turn
  • (Head) Hair
  • Facial Hair

JSON example of all facial-attributes sections specified

"facial_attributes": {
   "expression": [{
     "name": ["all"],
     "intensity": { "type": "range", "values": {"min": 0.0, "max": 1.0} },
     "percent": 100
   }],
   "gaze": [{
     "horizontal_angle": { "type": "range", "values": {"min": -30.0, "max": 30.0 }},
     "vertical_angle": { "type": "range", "values": {"min": -30.0, "max": 30.0 }},
     "percent": 100
   }],
   "head_turn": [{
     "pitch": { "type": "range", "values": {"min": -45.0, "max": 45.0 }},
     "yaw": { "type": "range", "values": {"min": -45.0, "max": 45.0 }},
     "roll": { "type": "range", "values": {"min": -45.0, "max": 45.0 }},
     "percent": 100
   }],
   "hair": [{
     "style": ["all"],
     "color": ["all"],
     "gender_matched_only": true,
     "ethnicity_matched_only": true,
     "relative_length": { "type": "range", "values": { "min": 0.5, "max": 1.0 }},
     "relative_density": { "type": "range", "values": { "min": 0.5, "max": 1.0 }},
     "percent": 100
   }],
   "facial_hair": [{
     "style": ["all"],
     "color": ["all"],
     "match_hair_color": true,
     "relative_length": { "type":  "list", "values": [1.0] },
     "relative_density": { "type": "list", "values": [1.0]},
     "percent": 100
   }]
 },


Expression

Expression manipulates the shape of the eyes, mouth, cheeks, etc. according to the name of an expression, and an amount, called intensity.

Note: at the moment, only a single expression can be applied at a time.

There are over 150 available expressions, which can be found our documentation appendix. The intensity is a '%' that denotes the relative deformation from neutral to each (full) expression.

name:

  • Default: ["none"]
  • Note: Use ["all"] as a shorthand for every option

intensity:

  • Default: 0.75
  • Min: 0
  • Max: 1

Two expressions with fully varying intensities

"expression": [{
  "name": ["eyes_closed_max", "chew"],
  "intensity": { "type": "range", "values": {"min": 0.0, "max": 1.0} },
  "percent": 100
}]

Visual examples of our expressions (see also: full input JSON to generate them):


Eye Gaze

The direction the eyes look, or our gaze value, is described from the -30 to 30 degrees as a difference from looking straight ahead. An eye looking up and to the left, from the character’s point of view, would have a positive horizontal angle and a positive vertical angle.

In the output, we also output the gaze_vector, which is the vector from the centroid of the eye volume, to the location of the pupil in 3d world-space coordinates.

horizontal_angle:

  • Default: 0
  • Min: -30
  • Max: 30

vertical_angle:

  • Default: 0
  • Min: -30
  • Max: 30

Gaze section featuring widest gaze ranges

"gaze": [{
  "horizontal_angle": { "type": "range", "values": {"min": -30.0, "max": 30.0 }},
  "vertical_angle": { "type": "range", "values": {"min": -30.0, "max": 30.0 }},
  "percent": 100
}]

Visual examples of gazes (see also: full input JSON to generate them):


Head Turn

Head turn describes the amount the head rotates about the neck/torso. It ranges from -45 to 45 degrees, in all 3 dimensions as a difference from posing level and straignt ahead. A head pitched/turned up and to the left, from the character’s point of view, would have a positive pitch angle and a positive yaw angle.

pitch:

  • Default: 0
  • Min: -45
  • Max: 45

yaw:

  • Default: 0
  • Min: -45
  • Max: 45

roll:

  • Default: 0
  • Min: -45
  • Max: 45

Head turn section featuring widest head-turn ranges

"head_turn": [{
  "pitch": { "type": "range", "values": {"min": -45.0, "max": 45.0 }},
  "yaw": { "type": "range", "values": {"min": -45.0, "max": 45.0 }},
  "roll": { "type": "range", "values": {"min": -45.0, "max": 45.0 }},
  "percent": 100
}]

Visual examples of head turns (see also: full input JSON to generate them):


Hair

Hair is grown procedurally based on a few parameters: named style, color, length, and density. Style names and color names can be found in the documentation appendix.

Note: We currently recommend limiting relative_length to greater than 0.5 and relative_density to greater than 0.7.

Note 2: At the moment, hair should not be specified if any headwear is specified.

Note 3: hair does not maintain the same color between renders for the same identity in a single job.

Note 4: The all keyword does not contain the none keyword, meaning that if all is requested for styles, colors, etc., a none or empty value will not be included - which means using the all keyword for hair returns zero bald heads.

Randomization color_seed controls variance in color of hair (and facial hair).

  • For consistent renders from one input to the next, always specify color_seed and set it to the same value
  • For randomized renders, specify color_seed as a range

You can also force that the hair style be gender-appropriate for the identity (gender_matched_only), and ethnically appropriate for the identity (ethnicity_matched_only). While these options may “feel” more appropriate, we have found that for machine learning, full variance can generalize better (use-case dependent).

style:

  • Default: ["none"]
  • Note: Use ["all"] as a shorthand for every option

color:

  • Default: ["chocolate_brown"]
  • Note: Use ["all"] as a shorthand for every option

relative_length:

  • Default: 1.0
  • Min: 0.5
  • Max: 1.0

relative_density:

  • Default: 1.0
  • Min: 0.5
  • Max: 1.0

color_seed:

  • Default: 0.0
  • Min: 0.0
  • Max: 1.0

gender_matched_only:

  • Default: true

ethnicity_matched_only:

  • Default:false
"hair": [{
  "style": ["all"],
  "color": ["all"],
  "gender_matched_only": true,
  "ethnicity_matched_only": true,
  "relative_length": { "type": "range", "values": { "min": 0.5, "max": 0.8 }},
  "relative_density": { "type": "range", "values": { "min": 0.5, "max": 1 }},
  "color_seed": { "type": "range", "values": { "min": 0, "max": 1 }},
  "percent": 100
}]

Visual examples of hairstyles and colors (see also: full input JSON to generate them):

Visual examples of hair lengths applied to a single style (see also: full input JSON to generate them):


Facial Hair

Facial hair is also grown procedurally based on the same parameter types as hair. There are mixes of just mustaches, just beards, and full mustache and beards. Check the documentation appendix for style and color names.

Note: At the moment, facial hair should not be specified if any mask is specified.

Note 2: The all keyword does not contain the none keyword, meaning that if all is requested for styles, colors, etc., a none or empty value will not be included.

Randomization color_seed controls variance in color of facial hair (and head hair).

  • For consistent renders from one input to the next, always specify color_seed and set it to the same value
  • For randomized renders, specify color_seed as a range

You can also force the facial hair color to be the same as the head hair color with the match_hair_color flag, for that particular render.

style:

  • Default: ["none"]
  • Note: Use ["all"] as a shorthand for every option

color:

  • Default: ["chocolate_brown"]
  • Note: Use ["all"] as a shorthand for every option

relative_length:

  • Default: 1.0
  • Min: 0.5
  • Max: 1.0

relative_density:

  • Default: 1.0
  • Min: 0.5
  • Max: 1.0

color_seed:

  • Default: 0.0
  • Min: 0.0
  • Max: 1.0

match_hair_color:

  • Default: true
"facial_hair": [{
  "style": ["all"],
  "color": ["all"],
  "match_hair_color": true,
  "relative_length": { "type":  "list", "values": [1.0] },
  "relative_density": { "type": "list", "values": [1.0]},
  "color_seed": { "type": "range", "values": { "min": 0, "max": 1 }},
  "percent": 100
}]

Visual examples of facial hair styles and colors (see also: full input JSON to generate them):

Visual examples of facial hair lengths applied to a single style (see also: full input JSON to generate them):


Accessories

The facial attributes section describes adornments that are additions beyond just the head anatomy.

This includes:

  • Glasses
  • Headwear/Hats
  • Face Masks
  • Headphones

An example of all accessories filled out:

"accessories": {
  "glasses": [{
    "style": ["all"],
    "lens_color": ["default"],
    "transparency": {"type":"range", "values": {"min":0.0,"max":1.0}},
    "metalness": {"type":"range", "values": {"min":0.0,"max":1.0}},
    "gender_matched_only": true,
    "percent": 100
  }],
  "headwear": [{
    "style": ["all"],
    "gender_matched_only": true,
    "percent": 100
  }],
  "masks": [{
    "style": ["all"],
    "percent": 100
  }],
  "headphones": [{
    "style": ["all"],
    "percent": 100
  }]
},


Glasses

Glasses have style names, several values for color, specified in the documentation appendix, transparency and "mirrored-ness", referred to as metalness. For clear glasses, the transparency value 1 is used. For fully mirrored glasses, the metalness value would be 1.

Note: The all keyword does not contain the none keyword, meaning that if all is requested for styles, lens colors, etc., a none or empty value will not be included.

style:

  • Default: ["none"]
  • Note: Use ["all"] as a shorthand for every option

lens_color:

  • Default: ["default"]
  • Note: Use ["all"] as a shorthand for every option

transparency:

  • Default: 1
  • Min: 0.0
  • Max: 1.0 (fully clear/transparent)

metalness:

  • Default: 1
  • Min: 0.0
  • Max: 1.0 (fully mirrored)

gender_matched_only:

  • Default: true

An example of clear, regular glasses json:

"glasses": [{
  "style": ["all"],
  "lens_color": ["default"],
  "metalness": {"type":"list", "values":[0]},
  "transparency": {"type":"list", "values":[1]},
  "gender_matched_only": true,
  "percent": 100
}]

Visual examples of glasses styles (see also: full input JSON to generate them):

Visual examples of glasses colors (see also: full input JSON to generate them):

Visual examples of glasses metalness (see also: full input JSON to generate them):

Visual examples of glasses transparency (see also: full input JSON to generate them):


Headwear

There are several styles of hats scaled to the size of the head. Simply a style name or list of them is specified. Style names can be found in the documentation appendix.

Note: At the moment, hair should not be specified if any headwear is specified.

Note 2: The all keyword does not contain the none keyword, meaning that if all is requested for styles, etc., a none or empty value will not be included.

You can also force the headwear to be gender-appropriate for the identity (gender_matched_only). Like most gender-appropriate values, while this option may “feel” more appropriate, we have found that for machine learning, full variance can generalize better (use-case dependent).

style

  • Default: ["none"]
  • Note: Use ["all"] as a shorthand for every option

gender_matched_only:

  • Default: true

All headwear style selection:

"headwear": [{
  "style": ["all"],
  "gender_matched_only": true,
  "percent": 100
}]

Visual examples of headwear styles (see also: full input JSON to generate them):


Masks

There are several styles of masks scaled to the size of the head. Simply a style name or list of them is specified. Each mask also has an off-nose version as a different style name. Style names can be found in the documentation appendix.

Position of the mask can be specified as well, defaulting to over the nose. Options include below the chin. Valid position values can be found in the documentation appendix

Note: At the moment, facial hair should not be specified if any mask is specified.

Note 2: The all keyword does not contain the none keyword, meaning that if all is requested for styles etc., a none or empty value will not be included.

You can also force the masks to be gender-appropriate for the identity (gender_matched_only). Like most gender-appropriate values, while this option may “feel” more appropriate, we have found that for machine learning, full variance can generalize better (use-case dependent).

Mask styles each have five variants, designated by values ranging from 0 to 4.

style:

  • Default: ["none"]
  • Note: Use ["all"] as a shorthand for every option

position:

  • Default: "2"
  • Note: Use ["all"] as a shorthand for every option

variant:

  • default: ["0"]
  • Note: Use ["all"] as a shorthand for every option

JSON for just selecting the floral pattern:

"masks": [{
  "style": ["ClothHead"],
  "percent": 100
}]

Visual examples of mask styles in position `2` (see also: full input JSON to generate them):

Visual examples of the `N95` style in different positions (see also: full input JSON to generate them):

Visual examples of the mask variants for all styles:


Headphones

There are several styles of headphones scaled to the size of the head. Simply a style name or list of them is specified. Style names can be found in the documentation appendix.

Note: At the moment, hats should not be specified if any headphones are specified.

style:

  • Default: ["none"]
  • Note: Use ["all"] as a shorthand for every option

Specifying a single headphone style to use:

"headphones": [{
  "style": ["0006_headsetMic"],
  "percent": 100
}]


Visual examples of headphones styles (see also: full input JSON to generate them):


Bodies [BETA]

NOTE: This feature is in beta and the JSON structure is subject to change. It is also not yet compatible with NIR

By default, bodies do not appear in generated images — only a head and a neck appear. To have bodies appear, the property enabled must be set as true.

The api provides bodies of varying shapes and sizes, and modifies clothing to fit over them. The default value for match_id_body_type is true. With default match_id_body_type, height and fat_content are automatically set and consistent for each ID — the settings are also the same for a given ID from one image to the next and from one job to the next. They are roughly mapped to the height and BMI of the identitity.

For machine learning and other uses requiring randomization of body shape and size, set match_id_body_type to false, and set height and associated fat_content. These may result in unusual proportions of head size relative to body size — testing to find a proper balance is recommended.

enabled:

  • Default: false

match_id_body_type:

  • Default: true

height:

  • Default: "median"

fat_content:

  • Default: "median"

percent:

  • Default: 100

You can check the documentation appendix for more info on valid height, valid fat_content, height logic, and bmi logic.

Body enabled only, which assigns default of matched to type:

"body": [{
  "enabled": true
}]

Body enabled and not matched, asking for full distros with all:

"body": [{
  "enabled": true,
  "match_id_body_type": false,
  "height": ["all"],
  "fat_content": ["all"],
  "percent": 100
}]

Visual examples of varying body fat content (see also: full input JSON to generate them):

Visual examples of varying body heights. Please note that the current camera location logic centers from the head, so changing heights is not immediately noticeable against some backgrounds, but can be seen here in the length of arm remaining in the image (see also: full input JSON to generate them):

Clothing 2.0 [BETA]

There are many clothing outfits available to choose from. Outfits are normally filtered to gender-normalized choices (for example: no dresses on males, no suit coats on females), but that filter can be dropped by changing the value of gender_matched_only to false. By default a gender-neutral outfit of jeans, t-shirt, and hoodie are applied if nothing is specified. Also, a new feature single_style_per_id has been added which will override a distibution by choosing a single outfit to apply to all of the renders for each ID selected.

NOTE: This feature is in beta and the JSON structure is subject to change.

outfit:

  • Default: "work_casual_4"

single_style_per_id:

  • Default: false

gender_matched_only:

  • Default: true

NOTE: The value of body must be set to enabled for clothing to appear.

Visual examples of clothing types:


Arms & Gestures

Arms [BETA]

The arms feature allows for percentage groups of arms configurations to be set. An arms configuration consists of individually set left and right arms, both disabled by default. At the minimum, an arm must be enabled to appear, and will be given a default gesture and rotation.

Arms have a wide range of motion along each axis. Arms each have a hand_gesture which can be specified individually or with the keyword all. The held_object for each hand can be specified, it can also include all, or be set to none.

Options for hand_gesture and held_object can be found in the documentation appendix.

Note: If a gesture and object combination occur with a gesture that cannot hold an item, the held_object will be set to none.

Note: Some choices of focal length, resolution, and camera depth in combination with arm rotation ranges can cause arms not to be visible in the frame when enabled.

Note: At the time being, enabling bodies creates an additional set of arms attached to the body. We recommend testing resolution, camera angles, and camera rotations if enabling bodies to prevent undesired results, like all sets of arms being visible simultaneously.

held_object_pose_seed sets the randomization for the variance of held objects.

  • For consistent renders from one input to the next, always specify held_object_pose_seed and set it to the same value
  • For randomized renders, specify held_object_pose_seed as a range

enabled:

  • Default: false

elbow_pitch:

  • Default: 0
  • Min: -90
  • Max: 90

elbow_yaw:

  • Default: 0
  • Min: -180
  • Max: 180

elbow_roll:

  • Default: 0
  • Min: -180
  • Max: 180

hand_gesture:

  • Default: ["relaxed_spread"]
  • Note: Check options here

held_object:

  • Default: ["none"]
  • Note: Check options here

held_object_pose_seed:

  • Default: 0
  • Min: 0
  • Max: 1

An example to start with:

{
    "render": {
        "resolution": {
            "w": 1020,
            "h": 720
        }
    },
    "faces": [
        {
            "identities": {
                "ids": [
                    80
                ],
                "renders_per_identity": 1
            },
            "arms": [
                {
                    "left": {
                        "enabled": true,
                        "elbow_pitch": {
                            "type": "list",
                            "values": [
                                55
                            ]
                        },
                        "elbow_yaw": {
                            "type": "list",
                            "values": [
                                15
                            ]
                        },
                        "elbow_roll": {
                            "type": "list",
                            "values": [
                                10
                            ]
                        },
                        "hand_gesture": [
                            "number_one"
                        ],
                        "held_object": [
                            "none"
                        ],
                        "held_object_pose_seed": {
                            "type": "list",
                            "values": [
                                0
                            ]
                        }
                    },
                    "right": {
                        "enabled": true,
                        "elbow_pitch": {
                            "type": "list",
                            "values": [
                                75
                            ]
                        },
                        "elbow_yaw": {
                            "type": "list",
                            "values": [
                                -10
                            ]
                        },
                        "elbow_roll": {
                            "type": "list",
                            "values": [
                                -10
                            ]
                        },
                        "hand_gesture": [
                            "hold_tool_thin"
                        ],
                        "held_object": [
                            "marker"
                        ],
                        "held_object_pose_seed": {
                            "type": "list",
                            "values": [
                                0
                            ]
                        }
                    },
                    "percent": 100
                }
            ],
            "camera": {
                "specifications": {
                    "focal_length": {
                        "type": "list",
                        "values": [
                            50
                        ]
                    }
                }
            }
        }
    ]
}

Full fields values with ranges

"arms": [
    {
        "left": {
            "enabled": true,
            "elbow_pitch": {
                "type": "range",
                "values": {
                    "min": -90,
                    "max": 90
                }
            },
            "elbow_yaw": {
                "type": "range",
                "values": {
                    "min": -180,
                    "max": 180
                }
            },
            "elbow_roll": {
                "type": "range",
                "values": {
                    "min": -180,
                    "max": 180
                }
            },
            "hand_gesture": [
                "all"
            ],
            "held_object": [
                "all",
            ],
            "held_object_pose_seed": {
                "type": "range",
                "values": {
                    "min": 0,
                    "max": 1
                }
            }
        },
        "right": {
            "enabled": true,
            "elbow_pitch": {
                "type": "range",
                "values": {
                    "min": -90,
                    "max": 90
                }
            },
            "elbow_yaw": {
                "type": "range",
                "values": {
                    "min": -180,
                    "max": 180
                }
            },
            "elbow_roll": {
                "type": "range",
                "values": {
                    "min": -180,
                    "max": 180
                }
            },
            "hand_gesture": [
                "all"
            ],
            "held_object": [
                "all",
            ],
            "held_object_pose_seed": {
                "type": "range",
                "values": {
                    "min": 0,
                    "max": 1
                }
            }
        },
        "percent": 100
    }
]

Visual examples of available hand gestures (right hand only) (see also: full input JSON to generate them)::

Visual examples of available held objects (right hand only):

Camera

The camera can both be placed in various locations, zoomed in/out, and use visual spectrum (vs) and near infrared (NIR) wavelengths.

A full example of the json is:

"camera": {
  "location": {
    "horizontal_angle": { "type": "range", "values": {"min": -100.0, "max": 100.0 }},
    "vertical_angle": { "type": "range", "values": {"min": -100.0, "max": 100.0 }},
    "offset_horizontal": { "type": "range", "values": {"min": -1.0, "max": 1.0 }},
    "offset_vertical": { "type": "range", "values": {"min": -1.0, "max": 1.0 }},
    "depth_meters": { "type": "range", "values": {"min": 0.2, "max": 1.5 }}
  },
  "specifications": {
    "focal_length": { "type": "range", "values": {"min": 1, "max": 300 } },
    "sensor_width": { "type": "list", "values": [4.6288] },
    "window_offset_horizontal": { "type": "list", "values": [0] },
    "window_offset_vertical": { "type": "list", "values": [0] },
    "lens_shift_horizontal": { "type": "list", "values": [0] },
    "lens_shift_vertical": { "type": "list", "values": [0] }
  },
  "wavelength_vs": {
    "percent": 90
  },
  "wavelength_nir": {
    "colocated_emitter": {
      "size_meters": 1.0,
      "intensity": 4.0
    },
    "percent": 10
  }
}


Location

The camera location is specified in relative degrees and a distance from the subject. Negative values are to the bottom and left of the subject. Distance is specified in meters, from the centroid of the volume of the head.

Without specifying any offsets, the centroid of the head is always at the center of the image. Horizontal and vertical offsets in meters are applied after the rotations, which can allow for the specification of images where the head is not centered.

horizontal_angle:

  • Default: 0.0
  • Min: -100.0
  • Max: 100.0

vertical_angle:

  • Default: 0.0
  • Min: -100.0
  • Max: 100.0

depth_meters:

  • Default: 1.0
  • Min: 0.2
  • Max: 5.0

offset_horizontal:

  • Default: 0.0
  • Min: -1.0
  • Max: 1.0

offset_vertical:

  • Default: 0.0
  • Min: -1.0
  • Max: 1.0

Camera above and to the right of the subject (no horizontal/vertical offset):

"location": {
  "horizontal_angle": { "type": "list", "values": [52]},
  "vertical_angle": { "type": "list", "values": [52]},
  "depth_meters": { "type": "list", "values": [0.1]}
}

Camera offset right and above the head:

"location": {
  "offset_horizontal": { "type": "list", "values": [0.05]},
  "offset_vertical": { "type": "list", "values": [0.05]},
  "depth_meters": { "type": "list", "values": [0.1]}
}

Visual examples of camera angles (see also: full input JSON to generate them):


Visual examples of camera depths with a fixed angle (see also: full input JSON to generate them):


Visual examples of camera offsets (see also: full input JSON to generate them):


Lens Specifications

The camera specifications section deals with the particular attributes of the lens and sensor, including:

  • Focal Length (Zoom)
  • Sensor Width
  • Lens Shift / Perspective
  • Window Offsets

The focal length (power/zoom) of the lens is specified in mm. We follow the standard as used my the vast majority of DSLR camera lenses and CGI rendering engines

focal_length:

  • Default: 100.0
  • Min: 1.0
  • Max: 300.0

sensor_width, measured in millimeters, is the "film back" or "back plate" of the optical sensor.

sensor_width:

  • Default: 33.0
  • Min: 0.001
  • Max: 1000.0

window_offset_horizontal and window_offset_vertical is a vray specific parameter related to the camera’s intrinsics, see: vray documentation

window_offset_horizontal and window_offset_vertical:

  • Default: 0
  • Min: -100
  • Max: 100

lens_shift_horizontal and lens_shift_vertical relate to Tilt-shift, see: wikipedia article

lens_shift_horizontal and lens_shift_vertical:

  • Default: 0
  • Min: -100
  • Max: 100

Speicying a zoom value of 18:

"specifications": {
  "focal_length": { "type": "list", "values": [18]},
  "sensor_width": { "type": "list", "values": [4.6288] }
}

Visual examples of camera focal lengths / zoom (see also: full input JSON to generate them):


Visual Spectrum & Near Infrared

Each entry of the faces array can have a proportion of the split in renders, just like percent groups. In addition, the NIR renders can specify an emitter with a given size & intensity.

wavelength_vs.percent:

  • Default: 100.0
  • Min: 0.0
  • Max: 100.0

wavelength_nir.percent:

  • Default: 0.0
  • Min: 0.0
  • Max: 100.0

colocated_emitter.size_meters:

  • Default: 1.0
  • Min: 0.0

colocated_emitter.intensity:

  • Default: 0.0
  • Min: 0.0
  • Note: try 4 to start

80% visual spectrum, 20% NIR renders:

"wavelength_vs": {
  "percent": 80
},
"wavelength_nir": {
  "colocated_emitter": {
    "size_meters": 1.0,
    "intensity": 4.0
  },
  "percent": 20
}

Visual examples of varying NIR emitter intensities (see also: full input JSON to generate them):

Visual examples of varying NIR emitter sizes (see also: full input JSON to generate them):


Environment

An environment surrounding the face is determined by an HDRI map, and zero to three directional lights in “percent groups”.

Varying HDRI intensity with three directional lights:

"environment": {
  "hdri": {
    "name": ["all"],
    "intensity": { "type": "range", "values": {"min": 0.5, "max": 2.0 }},
    "rotation": { "type": "range", "values": {"min": -180, "max": 180 }}
  },
  "lights": [{
    "light1": {
      "color_r": { "type": "list", "values": [1.0] },
      "color_g": { "type": "list", "values": [1.0] },
      "color_b": { "type": "list", "values": [1.0] },
      "rotation_x": { "type": "range", "values": {"min": 80.0, "max": 110.0} },
      "rotation_y": { "type": "list", "values": [0.0] },
      "intensity": { "type": "range", "values": {"min": 1.0, "max": 30.0} },
      "size_meters": { "type": "list", "values": [1.0, 2.0, 3.0, 4.0, 5.0] },
      "distance_meters": { "type": "list", "values": [0.5, 1.0, 2.0, 3.0] }

    },
    "light2": {
      "color_r": { "type": "list", "values": [1.0] },
      "color_g": { "type": "list", "values": [1.0] },
      "color_b": { "type": "list", "values": [1.0] },
      "rotation_x": { "type": "range", "values": {"min": 80.0, "max": 110.0} },
      "rotation_y": { "type": "list", "values": [0.0] },
      "intensity": { "type": "range", "values": {"min": 1.0, "max": 30.0} },
      "size_meters": { "type": "list", "values": [1.0, 2.0, 3.0, 4.0, 5.0] },
      "distance_meters": { "type": "list", "values": [0.5, 1.0, 2.0, 3.0] }
    },
    "light3": {
      "color_r": { "type": "list", "values": [1.0] },
      "color_g": { "type": "list", "values": [1.0] },
      "color_b": { "type": "list", "values": [1.0] },
      "rotation_x": { "type": "range", "values": {"min": -10.0, "max": 10.0} },
      "rotation_y": { "type": "range", "values": {"min": -30.0, "max": 30.0} },
      "intensity": { "type": "range", "values": {"min": 1.0, "max": 30.0} },
      "size_meters": { "type": "list", "values": [1.0, 2.0, 3.0, 4.0, 5.0] },
      "distance_meters": { "type": "list", "values": [0.5, 1.0, 2.0, 3.0] }
    },
    "percent": 100
  }]
},

HDRI Map

HDRI maps provide the primary lighting for the scene in terms of where the sun and surrounding lights are, and how bright they are, as well as color of the background. There are many to choose from, and they can also be rotated and increased/decreased in level/intensity to brighten the scene. For many use-cases, this is the only type of environment that needs to be specified.

name:

  • Default: ["lauter_waterfall"]
  • Note: Use ["all"] as a shorthand for every option

Higher intensity with static rotation:

"hdri": {
  "name": ["lauter_waterfall"],
  "intensity": { "type": "list", "values": [1.8]},
  "rotation": { "type": "list", "values": [-120]}
}

Visual examples of available HDRIs (see also: full input JSON to generate them):


Directional Lights

Between zero and three directional lights can be added to the scene to increase lighting variation on the face. They are passed in an array by specifying color, size, distance (from world origin), intensity, and rotation.

Up/down direction of lights is controlled by [code rotation_y] — [code -ve] moves a light down, [code +up] moves a light up. [code rotation_x] moves a light in a circle around the face — [code -ve] moves a light towards the right side of the face. When the value [code rotation_x = 0] is set, the light is at the left side of the face — when it is set to [code rotation_x = 90], the light is in front of the face and when set to [code rotation_x = 180], the light is at the right side of the face.

color_r:

  • Default: 255
  • Min: 0
  • Max: 255

color_g:

  • Default: 255
  • Min: 0
  • Max: 255

color_b:

  • Default: 255
  • Min: 0
  • Max: 255

rotation_x:

  • Default: 0.0
  • Min: -180.0
  • Max: 180.0

rotation_y:

  • Default: 0.0
  • Min: -180.0
  • Max: 180.0

intensity:

  • Default: 0
  • Min: 0
  • Max: 100

size_meters:

  • Default: 1.0
  • Min: 0.0
  • Max: 100.0

distance_meters:

  • Default: 1.5
  • Min: 0.0
  • Max: 100.0

Single white light with several locations/sizes/intensities:

"lights": [
  {
    "light1": {
      "color_r": { "type": "list", "values": [1.0] },
      "color_g": { "type": "list", "values": [1.0] },
      "color_b": { "type": "list", "values": [1.0] },
      "rotation_y": { "type": "range", "values": {"min": 80.0, "max": 110.0} },
      "rotation_x": { "type": "list", "values": [0.0] },
      "distance_meters": { "type": "list", "values": [0.5, 1.0, 2.0, 3.0] },
      "intensity": { "type": "range", "values": {"min": 1.0, "max": 30.0} },
      "size_meters": { "type": "list", "values": [1.0, 2.0, 3.0, 4.0, 5.0] }
    },
    "percent": 100
  }
]

Visual example of three lights (see also: full input JSON to generate them):


Inputs Reference

Fields Type Enum/Type Min Max Default
"render"
"resolution"
"w" number 64 4096 512
"h" number 64 4096 512
"engine"
string "vray" "vray"
"noise_threshold"
number 0.001 1 0.017
"denoise"
boolean true
"faces"
"identities"
"ids" array int [80]
"renders_per_identity"
integer 1 1000 1
"facial attributes"
"expression"
"name"
list Expressions "none"
"intensity"
list or range number 0 1 [0.75]
"gaze"
"horizontal_angle"
list or range number -30 30 [0]
"vertical_angle"
list or range number -30 30 [0]
"head_turn"
"pitch"
list or range number -45 45 [0]
"yaw" list or range number -45 45 [0]
"roll" list or range number -45 45 [0]
"hair"
"style"
list Styles "none"
"color"
list Colors "chocolate brown"
"color_seed"
list or range number [0]
"relative_length"
list or range number 0.5 1 [1.0]
"relative_density"
list or range number 0.5 1 [1.0]
"gender_matched_only"
bool true
"ethnicity_matched_only"
bool false
"facial_hair"
"style"
list Styles ["none"]
"color"
list Colors ["chocolate brown"]
"color_seed"
list or range number 0.0 1.0 [1.0]
"relative_length"
list or range number 0.5 1.0 [1.0]
"relative_density"
list or range number 0.5 1.0 [1.0]
"match_hair_color"
bool true
"accessories"
"glasses"
"style"
list Styles
"lens_color"
list Lens Colors
"transparency"
list or range number 0 1 [0]
"metalness"
list or range number 0 1 [1]
"gender_matched_only"
bool true
"headwear"
"style"
list Styles ["none"]
"gender_matched_only"
bool true
"masks"
"style"
list Styles ["none"]
"position"
list Positions ["2"]
"variant"
list Variants ["2"]
"headphones"
"style"
list Styles ["none"]
"camera"
"location"
"horizontal_angle"
list or range int -100 100 [0]
"vertical_angle"
list or range int -100 100 [0]
"depth_meters"
list or range number 0.2 5.0 [1.0]
"offset_horizontal"
list or range number -1.0 1.0 [0.0]
"offset_vertical"
list or range number -1.0 1.0 [0.0]
"specifications"
"focal_length"
list or range number 1.0 300.0 [100.0]
"sensor_width"
list or range number 0.0 1,000.0 [33.0]
"lens_shift_horizontal"
list or range number -100.0 100.0 [0.0]
"lens_shift_vertical"
list or range number -100.0 100.0 [0.0]
"window_offset_horizontal"
list or range number -100.0 100.0 [0.0]
"window_offset_vertical"
list or range number -100.0 100.0 [0.0]
"wavelength_vs"
"percent"
int 100
"wavelength_nir"
"colocated_emitter"
"size_meters" number 0 1
"intensity" number 4
"percent"
int 0
"environment"
"hdri"
"name"
list HDRIs ["lauter_waterfall"]
"intensity"
list or range number 0.1 5 [0.75]
"rotation"
list or range number -100 100 [0.0]
"lights"
"light1"
"color_r" list or range int 0 255 [255]
"color_g" list or range int 0 255 [255]
"color_b" list or range int 0 255 [255]
"rotation_x" list or range number 0 100 [0.0]
"rotation_y" list or range number 0 100 [0.0]
"rotation_z list or range number 0 100 [0.0]
"intensity" list or range number 0 100 [0.0]
"size_meters" list or range number 0 100 [0.25]
"distance_meters" list or range number 0 100 [1.5]
"light2"
"light3"
"geometry"
"body"
"enabled"
list bool ["false']
type" list Types ["2"]
"clothing"
"enabled"
list bool ["true']
"type" list Types ["3"]
"variant"
list Variants ["3"]

Full and Complex Example

A single image with most of the attributes set follows.

Note: headwear is not specified because we want hair.

View this JSON on GitHub.

{
  "render": {
    "resolution": {
      "w": 512,
      "h": 512
    }
  },
  "faces": [
    {
      "identities": {
        "ids": [75,80],
        "renders_per_identity": 50
      },
      "facial_attributes": {
        "expression": [{
          "name": ["all"],
          "intensity": { "type": "range", "values": {"min": 0.0, "max": 1.0} },
          "percent": 50
        },{
          "name": ["none"],
          "percent": 50
        }],
        "gaze": [{
          "horizontal_angle": { "type": "range", "values": {"min": -30.0, "max": 30.0 }},
          "vertical_angle": { "type": "range", "values": {"min": -30.0, "max": 30.0 }},
          "percent": 100
        }],
        "head_turn": [{
          "pitch": { "type": "range", "values": {"min": -45.0, "max": 45.0 }},
          "yaw": { "type": "range", "values": {"min": -45.0, "max": 45.0 }},
          "roll": { "type": "range", "values": {"min": -45.0, "max": 45.0 }},
          "percent": 25
        },{
          "pitch": { "type": "list", "values": [0.0]},
          "yaw": { "type": "list", "values": [0.0]},
          "roll": { "type": "list", "values": [0.0]},
          "percent": 75
        }],
        "hair": [{
          "style": ["all"],
          "color": ["all"],
          "gender_matched_only": true,
          "ethnicity_matched_only": true,
          "relative_length": { "type": "range", "values": { "min": 0.75, "max": 1.0 }},
          "relative_density": { "type": "range", "values": { "min": 0.75, "max": 1 }},
          "percent": 100
        }],
        "facial_hair": [{
          "style": ["none"],
          "percent": 100
        }]
      },
      "accessories": {
        "glasses": [{
          "style": ["all"],
          "lens_color": ["default"],
          "gender_matched_only": true,
          "percent": 25
        },{
          "style": ["all"],
          "lens_color": ["all"],
          "gender_matched_only": true,
          "percent": 10
        },{
          "style": ["none"],
          "percent": 65
        }],
        "headwear": [{
          "style": ["none"],
          "gender_matched_only": true,
          "percent": 100
        }],
        "masks": [{
          "style": ["all"],
          "percent": 25
        },{
          "style": ["none"],
          "percent": 75
        }],
        "headphones": [{
          "style": ["all"],
          "percent": 10
        },{
          "style": ["none"],
          "percent": 90
        }]
      },
      "environment": {
        "hdri": {
          "name": ["all"],
          "intensity": { "type": "range", "values": {"min": 0.5, "max": 2.0 }},
          "rotation": { "type": "range", "values": {"min": -180, "max": 180 }}
        },
        "lights": [{
          "light1": {
            "color_r": { "type": "list", "values": [1.0] },
            "color_g": { "type": "list", "values": [1.0] },
            "color_b": { "type": "list", "values": [1.0] },
            "rotation_x": { "type": "range", "values": {"min": 80.0, "max": 110.0} },
            "rotation_y": { "type": "list", "values": [0.0] },
            "intensity": { "type": "range", "values": {"min": 1.0, "max": 30.0} },
            "size_meters": { "type": "list", "values": [1.0, 2.0, 3.0, 4.0, 5.0] },
            "distance_meters": { "type": "list", "values": [0.5, 1.0, 2.0, 3.0] }
          },
          "light2": {
            "color_r": { "type": "list", "values": [1.0] },
            "color_g": { "type": "list", "values": [1.0] },
            "color_b": { "type": "list", "values": [1.0] },
            "rotation_x": { "type": "range", "values": {"min": 80.0, "max": 110.0} },
            "rotation_y": { "type": "list", "values": [0.0] },
            "intensity": { "type": "range", "values": {"min": 1.0, "max": 30.0} },
            "size_meters": { "type": "list", "values": [1.0, 2.0, 3.0, 4.0, 5.0] },
            "distance_meters": { "type": "list", "values": [0.5, 1.0, 2.0, 3.0] }
          },
          "light3": {
            "color_r": { "type": "list", "values": [1.0] },
            "color_g": { "type": "list", "values": [1.0] },
            "color_b": { "type": "list", "values": [1.0] },
            "rotation_x": { "type": "range", "values": {"min": -10.0, "max": 10.0} },
            "rotation_y": { "type": "range", "values": {"min": -30.0, "max": 30.0} },
            "intensity": { "type": "range", "values": {"min": 1.0, "max": 30.0} },
            "size_meters": { "type": "list", "values": [1.0, 2.0, 3.0, 4.0, 5.0] },
            "distance_meters": { "type": "list", "values": [0.5, 1.0, 2.0, 3.0] }
          },
          "percent": 100
        }]
      },
      "camera": {
        "location": {
          "horizontal_angle": { "type": "range", "values": {"min": -100.0, "max": 100.0 }},
          "vertical_angle": { "type": "range", "values": {"min": -100.0, "max": 100.0 }},
          "offset_horizontal": { "type": "range", "values": {"min": -0.02, "max": 0.2 }},
          "offset_vertical": { "type": "range", "values": {"min": -0.2, "max": 0.2 }},
          "depth_meters": { "type": "range", "values": {"min": 0.2, "max": 1.5 }}
        },
        "specifications": {
          "focal_length": { "type": "range", "values": {"min": 1, "max": 18 }}
        },
        "wavelength_vs": {
          "percent": 90
        },
        "wavelength_nir": {
          "colocated_emitter": {
            "size_meters": 1.0,
            "intensity": 4.0
          },
          "percent": 10
        }
      }
    }
  ]
}