mm.optimize package

Submodules

mm.optimize.camera module

This module contains functions related to finding the optimal orthographic and perspective transforms and camera matrices to align the 3DMM with source depth maps or images. They use sparse landmark correspondences between the 3DMM and the source to find an optimal initialization.

mm.optimize.camera.estimateCamMat(lm2D, lm3D, cam='orthographic')

Estimates camera matrix from 2D-3D landmark correspondences using the Direct linear transform / “Gold Standard Algorithm”.

For an orthographic camera, the algebraic and geometric errors in the algorithm are equivalent, so there is no need to do the least squares step at the end. The orthographic camera returns a 2x4 camera matrix, since the third row is just [0, 0, 0, 1].

Parameters:
  • lm2D (ndarray, (n, 2)) – landmark x, y coordinates in an image
  • lm3D (ndarray, (n, 3)) – landmark x, y, z coordinates in the 3DMM
  • cam (str) – ‘perspective’ or ‘orthographic’ to determine the type of camera projection matrix to return
Returns:

Camera projection matrix, (2, 4) if ‘orthrographic’, (3, 4) if ‘perspective’
Return type:

ndarray
mm.optimize.camera.initialRegistration(A, B)

Performs the Kabsch algorithm to find the optimal similarity transform parameters between 3D-3D landmark correspondences.

Parameters:
  • A (ndarray) – Set of source vertices
  • B (ndarray) – Set of target vertices such at B’ = s*R*A.T + t
Returns:

The optimal Euler angles, the 3D translation vector, and the scaling factor
Return type:

ndarray, (7,)
mm.optimize.camera.splitCamMat(P, cam='orthographic')

Splits the camera projection matrix into relevant intrinsic and extrinsic parameters.

Parameters:
  • P (ndarray) – A camera projection matrix
  • cam (str) – ‘perspective’ or ‘orthographic’ to determine how to split the camera projection matrix
Returns:

tuple containing:

K (ndarray (1,) or (3, 3)): orthographic scale parameter or perspective intrinsic camera matrix angles (ndarray (3,)): Euler angles t (ndarray (2,)): translation vector
Return type:

(tuple)

mm.optimize.depth module

This module contains functions to be used with the scipy.optimize package in order to fit the 3DMM to a target depth map.

mm.optimize.depth.initialShapeCost(param, target, model, w=(1, 1))
mm.optimize.depth.initialShapeGrad(param, target, model, w=(1, 1))
mm.optimize.depth.shapeCost(param, model, target, targetLandmarks, NN, w=(1, 1, 1), calcID=True)
mm.optimize.depth.shapeGrad(param, model, target, targetLandmarks, NN, w=(1, 1, 1), calcID=True)

mm.optimize.derivative module

This module contains helper functions that take the derivative of certain functions in the rendering pipeline.

mm.optimize.derivative.dR_dphi(angles)

Returns the derivative of the rotation matrix with respect to the z-axis rotation angle.

Parameters:angles (ndarray (3,)) – Euler angles
Returns:derivative of rotation matrix with respect to phi
Return type:ndarray, (3, 3)
mm.optimize.derivative.dR_dpsi(angles)

Returns the derivative of the rotation matrix with respect to the x-axis rotation angle.

Parameters:angles (ndarray (3,)) – Euler angles
Returns:derivative of rotation matrix with respect to psi
Return type:ndarray, (3, 3)
mm.optimize.derivative.dR_dtheta(angles)

Returns the derivative of the rotation matrix with respect to the y-axis rotation angle.

Parameters:angles (ndarray (3,)) – Euler angles
Returns:derivative of rotation matrix with respect to theta
Return type:ndarray, (3, 3)

mm.optimize.image module

mm.optimize.image.cameraShapeCost(param, model, lm2d, lm3dInd, cam)

Minimize L2-norm of landmark fitting residuals and regularization terms for shape parameters

mm.optimize.image.initialShapeCost(param, target, model, w=(1, 1))
mm.optimize.image.initialShapeGrad(param, target, model, w=(1, 1))
mm.optimize.image.textureCost(texCoef, img, vertexCoord, model, renderObj, w=(1, 1))
mm.optimize.image.textureGrad(texCoef, img, vertexCoord, model, renderObj, w=(1, 1))
mm.optimize.image.textureJacobian(texCoef, img, vertexCoord, model, renderObj, w=(1, 1), randomFaces=None)
mm.optimize.image.textureLightingCost(texParam, img, vertexCoord, sh, model, renderObj, w=(1, 1), option='tl', constCoef=None)

Energy formulation for fitting texture and spherical harmonic lighting coefficients

mm.optimize.image.textureLightingGrad(texParam, img, vertexCoord, sh, model, renderObj, w=(1, 1), option='tl', constCoef=None)
mm.optimize.image.textureLightingJacobian(texParam, img, vertexCoord, sh, model, renderObj, w=(1, 1), randomFaces=None)
mm.optimize.image.textureLightingResiduals(texParam, img, vertexCoord, sh, model, renderObj, w=(1, 1), randomFaces=None)

Energy formulation for fitting texture and spherical harmonic lighting coefficients

mm.optimize.image.textureResiduals(texCoef, img, vertexCoord, model, renderObj, w=(1, 1), randomFaces=None)

Module contents