{"id":439,"date":"2017-10-10T09:29:30","date_gmt":"2017-10-10T14:29:30","guid":{"rendered":"http:\/\/ciraldo.stream\/uwm-vfx2\/?page_id=439"},"modified":"2017-10-17T18:04:04","modified_gmt":"2017-10-17T23:04:04","slug":"face-tracking","status":"publish","type":"page","link":"https:\/\/ciraldo.stream\/uwm-vfx2\/topics\/face-tracking\/","title":{"rendered":"Face Tracking"},"content":{"rendered":"

Face Tracking<\/h1>\n

[ Lecture Notes<\/a> ] [ Lab<\/a> ] [ Homework<\/a> ]<\/p>\n

 <\/p>\n

Face Tracking<\/strong> is a special type of motion tracking. After Effects tracks various facial features<\/strong>\u00a0and structures,<\/strong> and can also calculate a few “measurements”<\/strong> based on the tracking data, such as “mouth open amount,”<\/strong> “eyes closed amount,”<\/strong> and others.<\/p>\n

With\u00a0Expression Controls<\/a> and a wee bit of expression script,<\/strong> we can also create some useful measurements of our own, such as “Face Center”<\/strong> or “Face Angle”<\/strong>.<\/p>\n

Uses For Facial Tracking<\/h2>\n

The facial tracking feature was added to After Effects in 2015, so it’s still relatively new to AE. It was added along with an early release of Adobe Character Animator<\/a> (which is an app used to animate cartoon characters faces<\/strong> in real-time<\/strong> using a webcam <\/strong>and an actor’s facial expressions, mouth movements, etc.)<\/p>\n

After Effects supports sending facial tracking data to Character Animator,<\/strong> so it seems this was one of their primary intended uses for Face Tracking. Even so, there are a few interesting things we can do with the feature strictly within AE, such as facial distortion, color correction, blemish removal, puppet pin control, and probably lots of things that haven’t even been thought of yet.<\/p>\n

How To Do The Tracking<\/h2>\n

It’s pretty easy to do the actual tracking part. This process will give lots<\/strong> of track point position keyframes<\/strong>\u00a0for facial features that we can use later on.<\/p>\n

    \n
  1. Add a footage layer,<\/strong> one with a person’s face in it.<\/li>\n
  2. Find a good\u00a0“rest pose,”<\/strong>\u00a0a frame of the footage in which the face\u00a0 as straight-facing and expressionless as possible.<\/li>\n
  3. Make a note<\/strong> of this frame by (a)<\/strong>\u00a0selecting the layer and then (b)<\/strong>\u00a0using the\u00a0Layer > Add Marker<\/strong> menu command. (Now you should have a little gray triangle marker on the layer to remember where the\u00a0rest pose<\/strong>\u00a0is.)<\/li>\n
  4. Create a<\/strong> rough mask<\/strong> around the face to track. It can be a simple mask shape like a circle. (You basically just need to let AE know where the face is.)<\/li>\n
  5. Select Animation > Track Mask<\/strong> from the menu, or right-click on the mask and select Track Mask.<\/strong> The Tracker Panel<\/strong> will open so you can operate the tracking.<\/li>\n
  6. In the Tracker Panel, choose\u00a0Face Tracking (Detailed Features)<\/strong> and then track forwards and backwards<\/strong> from the rest pose<\/strong> frame using the Tracker play controls.<\/li>\n
  7. AE will track the face.<\/strong> As it does so it will change your rough mask into one whose mask shape is auto-keyframed to tightly follow the contour of the face. Various tracking points<\/strong> will follow the movement of the eyes, nose, mouth, etc. A special-purpose effect called Face Track Points<\/strong> is added to the footage layer, and this is where the data from the track points gets stored<\/strong> as keyframes for later use.<\/li>\n<\/ol>\n

    What To Do With The Face Tracking Keyframes<\/h2>\n

    Snapchat Filter Stuff<\/h3>\n

    You can do some simple Snapchat-filter-esque things right away by adding an\u00a0expression<\/strong>\u00a0to\u00a0the property to be controlled<\/strong> and using the expression pickwhip<\/strong>\u00a0to grab the code for a track point<\/strong> such as Nose Tip<\/strong>\u00a0for example.<\/p>\n

    \"\"
    Clown Nose Position Linked To Nose Tip<\/figcaption><\/figure>\n

    Effect Control Points<\/h3>\n

    Some effects have one or more control points.<\/strong> For example, the\u00a0Bulge<\/strong> effect has a “bulge center”. You can place such effects directly on the\u00a0footage<\/strong> layer and link effect control points to track points as needed.<\/p>\n

    \"\"
    Bulge Center Linked To Nose Tip<\/figcaption><\/figure>\n

    Adjustment Layers For Effects With No Control Points<\/h3>\n

    For effects that don’t<\/strong> have control points (e.g., color correction stuff, blur, etc) you can add an Adjustment Layer<\/strong>\u00a0and link its position<\/strong> to a facial track point. Use a Mask<\/strong> on the Adjustment Layer to narrow its focus to the feature you’re tracking.<\/p>\n

    \"\"
    An Adjustment Layer’s Position Linked To Right Pupil<\/figcaption><\/figure>\n

    Slight Detour: Extract & Copy Face Measurements<\/h2>\n

    To get some more detailed info about what the actor’s face is doing, click on the mask and then return to the Tracker Panel. Go to your\u00a0Rest Pose<\/strong> frame that you marked earlier, and click the\u00a0Set Rest Pose<\/strong> button in the Tracker Panel. Then click the\u00a0Extract & Copy Face Measurements.<\/strong> This will create a second<\/em> special-purpose effect called\u00a0Face Measurements<\/strong> with all sorts of keyframed properties within.<\/p>\n

    \"\"
    Face Measurement Properties<\/figcaption><\/figure>\n

    Note that some of these properties seem like they should<\/strong> be<\/strong> in terms of pixels<\/strong> but are instead<\/strong> in terms of percentage.<\/strong>\u00a0<\/p>\n

    The 100% values are relative to the rest pose — pretty straightforward.<\/p>\n

    But the non-100% property values<\/strong> are not so straight-forward to use: they’re actually a percentage of the width, in pixels, <\/strong>between the outer corners of the left and right eyes.<\/strong>\u00a0We have the track points for the eyes, but how do we get the width between the points?<\/strong><\/p>\n

    Getting The Distance Between Two Points<\/h2>\n

    Let’s add a\u00a0Slider Control<\/strong>\u00a0effect to our tracked footage layer<\/strong> and call it\u00a0Eye Width.<\/strong><\/p>\n

    \"\"
    A Slider Control Renamed To “Eye Width”<\/figcaption><\/figure>\n

    Now add an expression to the Eye Width Slider property<\/strong>\u00a0and enter the following expression:<\/p>\n

    point1 = ; \/\/ Use expression pickwhip to grab the 1st point\r\npoint2 = ; \/\/ Same as above for the 2nd point\r\nlength(point1, point2);<\/pre>\n
    The expression pickwhip<\/strong> should be used (not the parenting pickwhip.<\/strong>) Place the typing insertion point between the equals sign<\/strong> and the semi-colon<\/strong>\u00a0before using the pickwhip.<\/p>\n
    \"\"
    Distance Between The Outer Corners Of The Eyes<\/figcaption><\/figure>\n
    With this\u00a0Eye Width<\/strong> property in place, we can now do some conversions between percentage and pixels from the Face Measurements<\/strong> effect!<\/p>\n
    Expression Controls For Intermediate, Reusable Numbers<\/h2>\n
    Now we’re starting to use expression controls to do some intermediate calculations on the face track points. Let’s create one for calculating the Mouth Offset Y<\/strong>\u00a0of the face. (This is the amount the mouth moves in the horizontal direction.)<\/p>\n
    Add a Slider Control and rename it to\u00a0Mouth Offset Y Pixels<\/strong>.<\/strong> Add the following expression to its Slider property:<\/strong><\/p>\n
    percentValue = ; \/\/ Pickwhip to a face measurement\r\neyeWidth = ; \/\/ Pickwhip to the Eye Width control\r\npercentValue * eyeWidth<\/pre>\n
    Then use the pickwhip to make it look like below:<\/p>\n
    \"\"
    Converting From Percentage To Pixels<\/figcaption><\/figure>\n
    Now we can use that\u00a0Mouth Offset X Pixels<\/strong> value elsewhere in our composition, perhaps to position a mustache.<\/strong><\/p>\n
    Angle Between Two Points<\/h2>\n
    Sometimes you have two tracked<\/strong> points<\/strong> with [x, y] position data, but you need to use them to control a rotation<\/strong> property.<\/p>\n
    This\u00a0<\/strong>expression, placed on any rotation property,<\/strong> will figure out the correct angle based on the two points you specify.<\/p>\n
    point1 = ; \/\/ <-- Pickwhip the 1st point\r\npoint2 = ; \/\/ <-- Same as above for second point\r\n\/\/ The following code uses trigonometry to calculate the angle.\r\ns = sub(point1, point2);\r\nradiansToDegrees( Math.atan2(s[1], s[0]) ) + 180;<\/pre>\n
    More Advanced Effects: Paint, Clone, Liquify, Mesh Deform<\/h2>\n
    Sometimes we need to stabilize the whole face or just a feature in order to apply more complicated effects. The process looks like this:<\/p>\n
      \n
    1. Stabilize Some Tracked Footage<\/li>\n
    2. Sub-Comp The Stabilized Footage<\/li>\n
    3. Apply The Effect To The Sub-Comp in an Outer Comp<\/li>\n
    4. Sub-Comp The Effects Work<\/li>\n
    5. Put the Effects-Work Sub-Comp in a Final Outer Composition<\/li>\n
    6. Reverse the stabilization in this final comp (put the facial movement back.)<\/li>\n<\/ol>\n
      Stabilize Face Position<\/h3>\n