CS 184: Computer Graphics Project Writeups

Homework 4: Clothsim

site: https://cal-cs184-student.github.io/hw-webpages-sp24-ashmchiu/hw4/

Overview

In this assignment, we create a physical simulation of a cloth. The cloth is able to maintain its shape and behavior through internal spring forces. It is also able to collide with external objects and itself. We also explore different shading options for the cloth, which includes diffuse and Phong shading, texture mapping, bump and displacement mapping, and ideal specular (mirror-like) environment mapping using cubemaps. Finally, we implement a few extra features, such as time-varying wind forces, a transparent blue-tinted shader and oscillating vertex shifter (to mimic a bouncing ball), and collisions with a new 3D primitive: cubes.

What we found interesting was the differing techniques for intersecting cloths with different primitives. In hindsight, it makes sense why we needed to perform collisions with them in different ways. It was fun to see all the different shaders in Part 5 come together–specifically, the texture map was really fun to add new textures into! It was also fun to see how this homework related back to Homework 2 with Phong shading.

We actually didn’t have difficult debugging journeys in this homework. Namely, something we were stuck on for a long time was bump and displacement mapping because our renders had much softer bumps and displacements than the references. However, we realized that this is because the default normal was 2 and height was 0.1 and when we updated this to be a normal of 100 and a height of 0.061, we could more clearly see the bumps and displacements.

Part 1: Masses and springs

In this part, our main goal was creating a grid of point masses and springs. To do so, we iterated through num_height_points and an inner loop of num_width_points to generate our point masses in row-major order. Depending on whether the orientation was horizontal or vertical, we either varied across the xz plane or the xy plane. Furthermore, if the point mass’s (x, y) index was within the cloth’s pinned vector, then we set their pinned boolean to true (which we’ll see at the corners of ../scene/pinned4.json).

Now that we’ve created our grid of point masses, we then created our springs with structural, shearing, and bending constraints. We note that

  • structural constraints were applied between a point mass and the point masses to its left and directly above it (if they existed)
  • shearing constraints were applied between a point mass and the point masses to its diagonal upper left and right (if they existed)
  • bending constraints were applied between a point mass and the point masses two to the left and two above it (if they existed).

Below, we’ve included several screenshots of ./clothsim -f ../scene/pinned2.json from various viewing angles to show the wireframe and structure of point masses and springs.

../scene/pinned2.json, entire grid in frame
../scene/pinned2.json, straight zoom in
../scene/pinned2.json, titled view of grid
../scene/pinned2.json, titled view zoom in

Now, here are some screenshots of ../scene/pinned2.json without any shearing constraints, with only shearing constraints, and with all constraints.

../scene/pinned2.json, without any shearing constraints
../scene/pinned2.json, with only shearing constraints
../scene/pinned2.json, with all constraints

Part 2: Simulation via numerical integration

Our main goal in this part is completing the Cloth::simulate method that runs one time step of time length delta_t and applies all accelerations uniformly to all point masses in the cloth.

Task 1: Compute total force acting on each point mass

First, we calculated all the external forces based on the external_accelerations, which just contains gravity as of now (see our wind simulations for an updated external_accelerations). Since we’re applying all accelerations uniformly, we can just sum them once, using Newton’s 2nd Law, \(F = ma\) (noting that the \(m\), mass, is constant). Then, we set each point mass’s forces to a Vector3D of our summed external forces.

Then, we needed to apply spring correction forces (based on our work in Part 1 in assigning spring constraints). We used Hooke’s law, \(F_s = k_s * (\vert\vert p_a - p_b\vert\vert - l)\), such that for each spring, we would apply \(F_s\) force to the point mass on one end, and then an equal, but opposite (negated) force to the other. For bending constraints, we set \(F_s\) to \(0.2\)x compared to shearing and structural constraints since they’re normally weaker.

Task 2: Use Verlet integration to compute new point mass positions

Next, we then computed a point mass’s new position since we now know the force acting on each point mass at a specific time step. For this, we use Verlet integration on all un-pinned vertices, calculating updated positions via

\[x_{t + dt} = x_t + (1 - d) * (x_t - x_{t - dt}) + a_t * dt^2\]

noting that \(d\) serves as a damping term given by the ClothParameters. We made sure in this step to also update the point mass’s last_position to track positions through time.

Task 3: Constrain position updates

Finally, to keep springs from being unreasonably deformed, we used the SIGGRAPH 1995 Provot paper to prevent springs from extending past than 10% of their rest_length at the end of any time step. If they did, we would correct the positions of the spring’s point masses:

  • If neither of the point masses were pinned, then we would performed half of the correct to each point mass.
  • If one of the point masses was pinned, we corrected fully by the other point mass.
  • If both of the point masses were pinned, we did nothing (because they couldn’t be moved :’)).

Below, we’ve included screenshots of ./clothsim -f ../scene/pinned4.json with default parameters, with both the wireframe and normal appearance.

../scene/pinned4.json, final resting state, wireframe
../scene/pinned4.json, final resting state, normal

Experimenting with parameters

In our cloth simulator, we have the ability to change the spring constant ks, the density, and damping constants. We’ll describe how the cloth differs when changing these to the default parameters. Below is the wireframe and normal appearance of ./clothsim -f ../scene/pinned2.json to show default parameters (ks = 5000 N/m, density = 15 g/cm^2, damping = 0.200000%).

../scene/pinned2.json, at rest, wireframe
default parameters
../scene/pinned2.json, at rest, normal
default parameters

Changing ks

While maintaining the default density = 15 g/cm^2 and damping = 0.200000%, let’s show ../scene/pinned2.json with ks = 50 N/m, ks = 500 N/m, and ks = 50000 N/m.

../scene/pinned2.json, at rest, wireframe
ks = 50 N/m, default density and damping
../scene/pinned2.json, at rest, normal
ks = 50 N/m, default density and damping
../scene/pinned2.json, at rest, wireframe
ks = 500 N/m, default density and damping
../scene/pinned2.json, at rest, normal
ks = 500 N/m, default density and damping
../scene/pinned2.json, at rest, wireframe
ks = 50,000 N/m, default density and damping
../scene/pinned2.json, at rest, normal
ks = 50,000 N/m, default density and damping

As ks increases, we see that the strength of the springs increase, which means that they bend much less and stay much flatter than when ks is smaller. Namely, this means that at our highest example of ks = 50,000 N/m, when the cloth is resting, there is little creasing (beyond to account for the pinning). Yet, in contrast, we see that when ks is smaller, the cloth is much more free and has many creases and folds at rest position. Since it’s much more free to move, even at rest position, with ks = 50 N/m, it is loose and wiggles: it doesn’t stay completely still.

When comparing how the cloth behaves as it moves from start to rest, we note that at lower ks values, because the spring constant is much less, the springs are less tight, so the cloth is a lot more flexible and waves as it swings down. In comparison, with higher ks values, the cloth is much more rigid and stays relatively flat as it swings down to rest position.

Changing density

While maintaining the default ks = 5000 N/m and damping = 0.200000%, let’s show ../scene/pinned2.json with density = 1 g/cm^2, density = 50 g/cm^2, density = 500 g/cm^2, and density = 5,000 g/cm^2.

../scene/pinned2.json, at rest, wireframe
density = 1 g/cm^2, default ks and damping
../scene/pinned2.json, at rest, normal
density = 1 g/cm^2, default ks and damping
../scene/pinned2.json, at rest, wireframe
density = 50 g/cm^2, default ks and damping
../scene/pinned2.json, at rest, normal
density = 50 g/cm^2, default ks and damping
../scene/pinned2.json, at rest, wireframe
density = 500 g/cm^2, default ks and damping
../scene/pinned2.json, at rest, normal
density = 500 g/cm^2, default ks and damping
../scene/pinned2.json, at rest, wireframe
density = 5,000 g/cm^2, default ks and damping
../scene/pinned2.json, at rest, normal
density = 5,000 g/cm^2, default ks and damping

We see that density operates almost inversely to ks. Namely, at the lowest density = 1 g/cm^2, we see the most rigid cloth, where there are less deformations in the cloth. Because at lower densities, the cloths will have lower mass, this means that the forces (that we accumulate) at each point mass will be less, and thus, means that there’s less forces pulling the cloth down, causing less wrinkles. In contrast, at our highest density = 5,000 g/cm^2, there are larger external forces working on the point masses (due to Newton’s 2nd Law). This means that the cloth weighs more, and thus, has many more wrinkles, which we can visibly see in the normal appearance with density = 5,000 g/cm^2.

When discussing how the cloth behaves as it moves from start to rest, we note that lower density values, the cloth weighs less, and as such, stays relatively flat as the forces acting against it aren’t as strong. This differs from using higher density values as the forces acting on the cloth are now stronger, and thus, the cloth deforms more, which causes more waves and wrinkles as it swings down.

Changing damping

While maintaining the default ks = 5000 N/m and density = 15 g/cm^2, let’s show ../scene/pinned2.json with damping = 0% and damping = 1%.

../scene/pinned2.json, moving, wireframe
damping = 0%, default ks and density
../scene/pinned2.json, moving, normal
damping = 0%, default ks and density
../scene/pinned2.json, moving, wireframe
damping - 1%, default ks and density
../scene/pinned2.json, moving, normal
damping = 1%, default ks and density

Here, we’ve opted to also include .gif depictions because what we believe to be more important is demonstrating the speed and flexibility at which the cloth moves.

../scene/pinned2.json, at rest, wireframe
damping = 0%, default ks and density
../scene/pinned2.json, at rest, normal
damping = 0%, default ks and density
../scene/pinned2.json, at rest, wireframe
damping = 1%, default ks and density
../scene/pinned2.json, at rest, normal
damping = 1%, default ks and density

Finally, we compare what happens when we mess with damping. At the lowest possible damping = 0% provided, we see that the cloth swings quite quickly back and forth. In contrast, at the highest possible damping = 1%, the cloth moves really quickly and seemingly stops at the rest position (that matches the rest position with default parameters). Since we know that damping messes with the velocity term in Verlet integration, with no damping, this means there is no loss of energy due to friction, heat loss, or any other force. Therefore, the cloth just swings back and forth, with wrinkling and no rigid structure because there is no loss of energy. However, with the highest damping, the cloth falls much slower and holds its structure a bit better. The expalanation for this is that the forces acting on it (namely, gravity) are now dampened, which means that the positions of the point masses don’t move as quickly. This reflects a state in which there is a loss of energy, so not all the forces that are thrust upon the cloth actually convert into energy that moves the cloth–some is lost or dissipated.

Part 3: Handling collisions with other objects

Throughout this part, we are colliding a cloth with spheres and planes. We want to make the cloth collide in a realstic manner–which we will later elevate in Part 4 with self collisions!

Task 1: Handling collisions with spheres

In handling sphere collisions, we implemented Sphere::collide, noting that if the position of the point mass would be within the sphere, we would “bump” it to the surface of the sphere. To do so, we first used Euclidean distances to check whether the point mass was inside the sphere, returning if it wasn’t. Then, we computed the tangent point along the sphere where the collision would have occurred, using this to compute the correction vector and applying that correction vector to the point mass’s last_position, scaled by 1 - f where f is friction.

We also updated Cloth::simulate, checking for collisions between every PointMass and every possible CollisionObject, of which Sphere was a type.

Below, we’ve included screenshots of ./clothsim -f ../scene/sphere.json in its final resting state, using the default ks = 5000, as well as ks = 500 and ks = 50000.

../scene/sphere.json, final resting state, wireframe,
default ks = 5000
../scene/sphere.json, final resting state, normal,
default ks = 5000
../scene/sphere.json, final resting state, wireframe,
ks = 500
../scene/sphere.json, final resting state, normal,
ks = 500
../scene/sphere.json, final resting state, wireframe,
ks = 50000
../scene/sphere.json, final resting state, normal,
ks = 50000

We see that when ks increases, the cloth becomes more rigid and the folds structured and hence, there are also less folds. In contrast, when ks decreases, the cloth becomes much more flexible, and we can see that the folds around the sphere increase as the cloth as able to mold to the sphere more. We can see that at lower ks values, the cloth drapes more willingly, so we can see the spherical structure more whereas at the highest ks value, the creasing at the top obscures the sphere’s smoothness around the sides.

In discussion regarding the purpose of ks, this does make sense as a stronger/larger spring constant means that the spring is stronger, so the cloth holds shape stronger (and vice versa for the smaller ks).

Task 2: Handling collisions with planes

To handle collisions with planes, we implemented the Plane::collide method. Similarly, we first checked whether a point had crossed the plane by checking whether the dot product between the normal vector of the plane and the point mass’s last_position and position had different signs. If the signs were the same, this meant that the point mass had not passed the plane, so we directly returned.

Otherwise, we again calcualted the tangent point at which the collision between the point mass and the plane would have occurred, using that to compute the correction vector (in which we factored in the SURFACE_OFFSET by the normal vector of the Plane). Then, similarly to Sphere::collide, we applied the correction vector to the point mass’s last_position, scaled by 1 - f where f is friction.

We also updated Cloth::simulate, checking for collisions between every PointMass and every possible CollisionObject, of which Plane was a type.

Here’s a few screenshots of ./clothsim -f ../scene/plane.json, lying peacefully at rest on the plane. (We are kalm – no panic!)

../scene/plane.json, final resting state, wireframe
../scene/plane.json, final resting state, normal
../scene/plane.json, final resting state, mirror
../scene/plane.json, final resting state, custom texture (eddie)

Part 4: Handling self-collisions

In Part 3, we handled collisions between a cloth and other objects, however, if a cloth were to collide with itself right now, it would have no comprehension of that collision (and just clip through itself). What we aim to implement (and what we did!) was prevent this clipping, ensuring that if the cloth fell on itself, it would fold. Below, we run ./clothsim -f ../scene/selfCollision.json prior to implementing Task 4 and after. To the left, you can see the clipping, particularly the glitching after the cloth is fully on the plane (no longer resting–this is a panic!). To the right, you can see the cloth folding over itself and resting in a much more peaceful way.

A very send help moment.
Help was sent.

To perform this task, we implemented spatial hashing, mapping floats to a vector<PointMass *>, where each float uniquely represented a 3D box volume in the scene and the vector<PointMass *> was a vector containing all the point masses in that 3D box volume. Using this hash table, we would apply a repulsive collision force if any pair of point masses in the same 3D box volume got too close to one another.

Task 1: Cloth::hash_position

First, we implemented Cloth::hash_position to take in a point mass’s position and to effectively calculate its 3D box volume’s index in our spatial hash table. To do so, we calculated a (w, h, t) such that w = 3 * width / num_width_points, h = 3 * height / num_height_points, and t = max(w, h). From here, we determined what 3D box the point was in by calculating x = floor(pos.x / w) * w, and similarly for the y and z axes. Knowing this, we transformed this 3D position into a 1D position by using a formmulation similar to how we determined indices for supersampled points in Homework 1, returning this value.

Task 2: Cloth::build_spatial_map

Now, since we had a way of calculating each point mass’s hash position, we can now construct our spatial map. To do so, we iterated over all point_masses in the cloth and found its hash_position. Then, since at each key-value pair in the map, our value is a vector<PointMass *>, we initialize a new vector<PointMass *> if there isn’t already one at the hash position, and then we push_back the current point mass.

Task 3: Cloth::self_collide

Finally, we implemented our full self-collision method. For the passed in point mass, we check whether the normalized distance between it and all candidate point masses is less than \(2 * thickness\), and also whether it’s normalized distance is greater than 0 (to prevent a point mass from colliding with itself). Then, we would calculate the final correction to be unit distance between the two point masses multipled by the correction to ensure that the pair would be \(2 * thickness\) distance apart.

Then, we updated Cloth::simulate, first to call our build_spatial_map function. Then, we iterated through all point_masses, calling self_collide on it.

Below are 6 screenshots of ./clothsim -f ../scene/selfCollision.json to show how the cloth falls and folds on itself.

../scene/plane.json, early, initial self-collision
../scene/selfCollision.json, continuing folding over itself
../scene/selfCollision.json, deep into self-collisions
../scene/selfCollision.json, beginning to flatten out
../scene/selfCollision.json, smoothing out
../scene/selfCollision.json, final resting state

Now, we’re going to be experimenting with ks and density values. As a reference, here is a .gif of the self-collision in ./clothsim -f ../scene/selfCollision.json with default ks = 5000 N/m and density = 15 g/cm^2.

../scene/selfCollision.json,
default ks = 5000 N/m, density = 15 g/cm^2

Experimenting with ks

Here are .gif files of the self-collision in ./clothsim -f ../scene/selfCollision.json with default density = 15 g/cm^2, but varying ks with a low ks = 50 N/m and a high ks = 50,000 N/m.

../scene/selfCollision.json, low ks = 50 N/m
../scene/selfCollision.json, high ks = 50,000 N/m

We can see through these .gif files and the below screenshots that at smaller ks values, such as the ks = 50 N/m provided, the cloth folds in a much more rippling fashion, with each fold being smaller. This allows overall for more self-collisions, because at a lower spring constant, the cloth doesn’t hold as much rigid structure, so it is flexible to fold a lot. In contrast, with a higher ks = 50,000 N/m, we see that since the spring constant is higher, the cloth is tighter and more rigid, holding its structure, which means that it folds a lot less against itself, so there are less self-collisions and overall, less wrinkles. There are gaps between the layers of the cloth as it lays on top of other parts of itself, which we can see more in the second and third screenshots for ks = 50,000 N/m.

To the left, we’ve included screenshots of how the cloth behaves with ks = 50 N/m as it falls on itself while on the right are screenshots of how the cloth behaves with ks = 50,000 N/m.

../scene/selfCollision.json,
ks = 50 N/m, beginning to ripple
../scene/selfCollision.json,
ks = 50,000 N/m, small waves
../scene/selfCollision.json,
ks = 50 N/m, rippling more
../scene/selfCollision.json, ks = 50,000 N/m, more small waves
../scene/selfCollision.json,
ks = 50 N/m, beginning to settle
../scene/selfCollision.json, ks = 50,000 N/m, beginning to settle
../scene/selfCollision.json,
ks = 50 N/m, resting state
../scene/selfCollision.json, ks = 50,000 N/m, resting state

Experimenting with density

Here are .gif files of the self-collision in ./clothsim -f ../scene/selfCollision.json with default ks = 5000 N/m, but varying density with a low density = 1 g/cm^2 and a high density = 50 g/cm^2.

../scene/selfCollision.json, low density = 1 g/cm^2
../scene/selfCollision.json, high density = 50 g/cm^2

We can see through these .gif files and the below screenshots that at smaller density values, such as the density = 1 g/cm^2 provided, the cloth almost seems bouncier, holding its structure and making larger waves. This means there are less self-collisions: this makes sense because at a lower density, this means that the mass is less, so the overall force applied to the cloth is less, making it collide less. In contrast, with a higher density = 50 g/cm^2, we see that the cloth folds a lot more, rippling as it falls on itself. Compared to smaller densities, each fold in the cloth is smaller. At a higher density, this makes sense because larger density means larger mass, and as such, there is a larger force applied to it, allowing for more self-collisions.

To the left, we’ve included screenshots of how the cloth behaves with density = 1 g/cm^2 as it falls on itself while on the right are screenshots of how the cloth behaves with density = 50 g/cm^2.

../scene/selfCollision.json,
density = 1 g/cm^2, beginning to ripple
../scene/selfCollision.json,
density = 50 g/cm^2, small waves
../scene/selfCollision.json,
density = 1 g/cm^2, rippling more
../scene/selfCollision.json, density = 50 g/cm^2, more small waves
../scene/selfCollision.json,
density = 1 g/cm^2, beginning to settle
../scene/selfCollision.json, density = 50 g/cm^2, beginning to settle
../scene/selfCollision.json,
density = 1 g/cm^2, resting state
../scene/selfCollision.json, density = 50 g/cm^2, resting state

Part 5: Shaders

A shader is a program that we can use in order to speed up our rendering processes by moving the workload of rendering from the CPU to the GPU. While we did not use any advanced GPU functionality in this homework, the GLSL shaders that we used in this homework was able to speed up the rendering process tremendously, being able to quickly calculate the effects of different lighting conditions and materials.

The vertex shader works by applying transforms to vertices, which is useful for displacement mapping and other types of transforms where vertex positions need to be manipulated. The vertex shader produces fragments, which are then used as the input for the fragment shader. On the other hand, the fragment shader takes in the fragments generated from the vertex shader and outputs a single 4D vector for each pixel, namely an RGBA value. The fragment shader accounts for how different material types and geometries interact with light. Using both the vertex shader and the fragment shader, the CPU is able to offload a large chunk of the graphics pipeline onto the GPU, where it can be parallelized and computed quickly for real-time rendering.

Task 1: Diffuse Shading

Implementing the main method of Diffuse.frag, our main goal was to recreate diffuse lighting using the formula

\[L_d = k_d (I/r^2)\max(0, n \cdot l)\]

We did this by calculating the radius from the light position to the vertex, calculating the normalized normal vector, and normalizing the light vector. From there, we set the out_color, noting that we are given \(k_d = 1\) to \(L_d\) as given in the diffuse lighting formula.

Below is a screenshot of running ./clothsim -f ../scene/sphere.json with diffuse shading.

../scene/sphere.json, diffuse shading

Task 2: Blinn-Phong Shading

From lecture, we know that Blinn-Phong shading outputs a light with the equation

\[L = k_a I_a + k_d (I/r^2) \max(0, n \cdot l) + k_s (I/r^2) \max(0, n \cdot h)^p\]

effectively, the addition of ambient, diffuse, and specular lighting (which reflect each of the three terms in the equation above). We note that ambient lighting is shading that doesn’t depend on anything (it is constant). As a whole, Blinn-Phong shading allows us to account for not only the angle between each vertex and a light source (diffuse), but also from what perspective and viewing angle the camera is from (specular). Namely, the coolest part of Blinn-Phong shading (in our opinion), is that in specular shading, it uses the half-vector between the viewing direction and the light source, which is used to approximate where the maximum specular reflection (the greatest intensity reflection) is. Blinn-Phong is faster than the methods we took in Part 3 to light up scenes.

Building off our work in Diffuse.frag, we also now need to compute ambient and specular shading.

For ambient shading, we choose k_a = 1 and I_a = vec(1/61, 1/161, 1/61). Since the formula is \(L_a = k_a I_a\), this is all we need! With only purely ambient shading, running ./clothsim -f ../scene/sphere.json gives

../scene/sphere.json, ambient shading

Then, we reused our code from Diffuse.frag for diffuse shading, \(L_d\). With only purely diffuse shading, running ./clothsim -f ../scene/sphere.json gives

../scene/sphere.json, diffuse shading

Finally, we add our specular highlights! To do so, we calculate the viewing vector v as the difference between the camera position and the vertex position. We then calculate the half-vector between v and the normal vector, normalizing the half-vector. Finally, we calculate our specular shading using the formula \(L_s = k_s (I / r^2) * \max(0, n \cdot h)^p\). We use \(k_s = 1\) and \(p = 161\). With only purely specular shading, running ./clothsim -f ../scene/sphere.json gives

../scene/sphere.json, specular shading

We see the main specular point near the center and front of the sphere.

Now, let’s put it all together. We know that Blinn-Phong reflections are just a summation of ambient, diffuse, and specular shading. Therefore, we get our resulting \(L = L_a + L_d + L_s\), which when running ./clothsim -f ../scene/sphere.json, gives

../scene/sphere.json, entire Blinn-Phong model

Task 3: Texture Mapping

Modifying Texture.frag, we set the output single-4 dimensional vector to be a call to the built-in function texture(sampler2D tex, vec2 uv), passing in the texture map provided and the texture space coordinate v_uv.

Below, we include screenshots of running ./clothsim -f ../scene/sphere.json with the default Campanile texture map and :eddie-proctor-vidoe:, one of our favorite Slackmojis.

../scene/sphere.json, default Campanile texture
../scene/sphere.json, custom texture

For reference, here is :eddie_proctor_vidoe: (yes, the spelling is correct) in all its glory:

:eddie_proctor_vidoe:

Task 4: Displacement and Bump Mapping

In this task, we implement both displacement and bump mapping, two different ways of representing surface roughness. The first part of this task is bump mapping, where the texture map stores height information. The height information gleaned from the texture map is used to calculate the displaced model space normal $n_d$ from the original normal vector inside Bump.frag.

For bump mapping, in Bump.frag, we implemented our h(vec2 uv) function to return the r-value of the return of calling texture. We also used the provided normal and tangent to calculate the bitangent = cross(normal, tangent) to create the TBN matrix as

\[TBN = \begin{bmatrix}t & b & n\end{bmatrix}\]

The displaced model space normal is calculated by first calculating the differential height via

\[dU = (h(u + \frac{1}{w}, v) - h(u, v)) \times k_h \times k_n\] \[dV = (h(u, v + \frac{1}{h}) - h(u, v) \times k_h \times k_n)\]

where $k_h$ and $k_n$ are height and normal scaling factors, respectively, and $h$ is a function that returns the value of the texture map at the passed in coordinates.

Finally, we can apply the TBN matrix to the local space normal $n_o = (-dU, -dV, 1)$ to get $n_d$. This result is like calculating the normal vector to the object had its surface been displaced by the texture map height location. Importantly, though, the surface is not displaced in bump mapping, and the object maintains its original geometry. We reuse our code from Phong.frag as our shading calculations, replacing our normal vector with our calculated \(n_d\).

Below are screenshots of running ./clothsim -f ../scene/sphere.json with bump mapping, setting the normal to 100 and height to 0.061. We use texture3.png.

../scene/sphere.json, bump mapping on sphere
../scene/sphere.json, bump mapping on cloth
../scene/sphere.json, bump mapping on sphere and cloth
../scene/sphere.json, bump mapping with cloth over sphere

For displacement mapping, we copied over the work we did in Bump.frag into Displacement.frag. Then, we updated Displacement.vert to calculate v_position as the old position u_model * in_position summed with the original model space vertex normal normalize(u_model * in_normal) scaled by h(in_uv) * u_height_scaling. Because gl_position represents the screen space position, we also modified this to reflect the new position.

Below are screenshots of running ./clothsim -f ../scene/sphere.json with displacement mapping, setting the normal to 100 and height to 0.061. We use texture3.png.

../scene/sphere.json, displacement mapping on sphere
../scene/sphere.json, displacement mapping on cloth
../scene/sphere.json, displacement mapping on sphere and cloth
../scene/sphere.json, displacement mapping with cloth over sphere

Comparing Bump and Displacement Mapping

../scene/sphere.json, bump mapping on sphere
../scene/sphere.json, displacement mapping on sphere
../scene/sphere.json, bump mapping with cloth over sphere
../scene/sphere.json, displacement mapping with cloth over sphere

Looking at these images where the only difference is using bump mapping versus displacement mapping, we see that bump mapping maintains the smoothness of the sphere while displacement mapping, while actually changing the location of each vertex, no longer is smooth. Namely, we see that bump mapping mainly modifies the fragments, not the locations of the vertices themselves across screen space, so it gives an illusion of bumps and highlights and indentations, by creating highlights where the brick highlights would be through the reuse of Phong shading. On the other hand, we see that displacement mapping actually moves around the vertices of the sphere based on the texture map , matching the roughness of the bricks, deforming from the perfect smooth texture the sphere had.

Playing around with coarseness, here are screenshots of running ./clothsim -f ../scene/sphere.json with bump displacement mapping, setting the normal to 100 and height to 0.061. We use texture3.png. On the top row, we use ./clothsim -f ../scene/sphere.json -o 16 -a 16 for lower resolution and on the bottom row, we use ./clothsim -f ../scene/sphere.json -o 128 -a 128 for higher resolution.

../scene/sphere.json, bump mapping
-o 16 -a 16
../scene/sphere.json, displacement mapping
-o 16 -a 16
../scene/sphere.json, bump mapping
-o 128 -a 128
../scene/sphere.json, displacement mapping
-o 128 -a 128

Now, when we compare the coarseness of the sphere while changing the resolution of the image, the displacement mapping affects the coarseness of the sphere more than bump mapping does. Namely, we see that for the smaller resolution image (lower coarseness), there’s less sampling points across the sphere for the displacement map to change the vertices of, so we can clearly see the jumps across different vertices (there are extreme jagged edges across the sphere and it is more spikey than it is smooth). In contrast, at a higher resolution, the coarseness is higher, and the surface texture of the sphere is much sharper with displacement mapping. More vertices have modified positions due to the displacement mapping so the texture seems more realistic and reflects the texture of the brick better. Because there are more vertices whose positions are displaced, although the displacement mapping sphere is still spikey, the overall roundness of the sphere is maintained.

While displacement mapping has large differences across different resolutions, bump mapping does not. Because bump mapping doesn’t actually change the position of vertices (thus, changing the shape of the sphere), and instead just maps the texture points onto the sphere, the surface of the sphere doesn’t change with changing resolutions.

Task 5: Environment-mapped Reflections

Our job in this task is to update Mirror.frag. We first calculate the outgoing eye-ray, \(w_o\) by subtracting the camera’s position, u_cam_pos from the fragment’s position, v_position. Then, using the surface normal vector, we calculate the incoming \(w_i\) by calculating \(w_o - 2 (w_o \cdot n) * n\). Finally, we sample the texture for the incoming direction \(w_i\) calculated.

Below are screenshots of running ./clothsim -f ../scene/sphere.json with environment-mapped reflections.

../scene/sphere.json, mirror shader on sphere
../scene/sphere.json, mirror shader on cloth
../scene/sphere.json, mirror shader on sphere and cloth
../scene/sphere.json, mirror shader with cloth over sphere

Custom Shader: Jerover Blue, So True

We then implemented a custom shader in Custom.frag that was a hybrid between mirror and phong shading. We added an extra interpolation between the color from mirror shading and interpolated that with the color Jerover Blue, interpolating with an alpha value using Schlick’s approximation. Finally, in the out_color, we set \(\alpha\) to be 0.5 to create a transparent output.

The resulting shader creates a blue-tinted transparent plastic material, shown by the images of ./clothsim -f ../scene/sphere.json below.

../scene/sphere.json, double reflected bridge
../scene/sphere.json, mirror and phong combination
../scene/sphere.json, where did the sphere go?
../scene/sphere.json, the sphere is still here

The transparency that we mimicked in this shader almost makes it look like the sphere is missing when the cloth is put over it! However, when looking from the underside, we can see the sphere almost like a crystal orb.

Building on this, we also modified Custom.vert to displace the vertices of the sphere to simulate a bouncy/non-uniform sphere. We used a random-number generator, making oscillating displacements using a combination of sin and cos to displace each vertex at each axis. We also used a few magic numbers in our displacement calculations, fit to our liking for bounciness.

bouncy wheeeee!

Part 6: Extra credit

For extra credit, we implemented wind, a custom shader, and collisions with cubes.

Whoosh! (It’s windy out here)

We decided to add wind to our simulation by adding it into the external_accelerations, much like gravity. However unlike gravity, we wanted our wind to be variable at different timesteps in the simulation, as some static wind force would cause the cloth to settle into some equilibrium position and appear unnatural.

We tried a few approaches in order to simulate this behavior, the first of which was to try and simulate osciallation with sine and cosine functions. However, we found this to be too predictable and did not feel like natural wind behavior.

The next approach we used, which ended up being our final result, was empirically found and relied heavily on randomness. The idea is that the wind value is limited between 0 and a certain max value. If the current value for wind acceleration is near the mean value of the range, on the next iteration it would be encouraged to move towards the extremities. If the current value is near the extremities of the range, it would be encouraged to move towards the middle of the range. Of course, this is all probabilistic, with a certain degree of randomness injected into every calculation.

Below are screenshots of running ./clothsim -f ../scene/pinned2.json while changing the wind value.

../scene/pinned2.json, positive wind value
../scene/sphere.json, negative wind value

and you can see here that negative wind pushes the cloth towards the right while positive wind pushes it to the left.

We also include here a .gif of dynamically updating the wind values and its effect on a pinned cloth.

whoosh!

Custom Shader

Our custom shader is described in Part 5.

Here, we’ve combined our wind and custom fragment shader (not including the vertex shifts) in the following .gif of running ./clothsim -f ../scene/sphere.json.

jerover whoosh!

Collisions with Cubes

We implemented a methodology of colliding a cloth with cubes. To do so, we originally actually created a new cube.h and cube.cpp file, but struggled with defining all 36 vertices (and 12 triangles) in a way that made sense for surface normals, so we opted instead to add to plane.h to create two new attributes called scale and axis_aligned to constrain the plane (rather than let it be infinite) and constructed our cube out of six planes. This allowed us to define each Plane’s normal (since this was already a part of the Plane struct), and we effectively just collided the cloth with each of the size Planes that constructed this cube.

To actually perform the collision, we needed to modify Plane::collide, checking whether we were in a cube setting before setting extra constraints that if the current position of the PointMass was outside the finite plane, then we could just exit as no collision has occurred. We also modified Plane::render to account for the size of the cube (since we allowed in the .json files for customization of the size of the cube).

Here, we show a collision between our cube and the cloth.

../scene/cube.json, collision with a cube

We also have a scene here with our cube, a sphere, and a plane. We can see the cloth colliding with all three in this image.

../scene/cubesphereplane.json, collision with a cube, sphere, and plane

In both these images, we can see that the cloth slips through the edges of the cube. The reasoning for this is because only the PointMasses on the cloth are actually colliding with the cube, rather than the Springs connecting the PointMasses, and hence, the Springs do cut into the cube unnaturally, but the PointMasses clearly do not.

Contributors

Edward Park, Ashley Chiu

We really enjoyed working on this project :) We worked mainly together in person for Parts 1-5. Then spring break started, so we worked asynchronously, keeping each other up on what progress we made (specifically since we only had extra credit left). For extra credit, we split it up so Eddie worked on wind while Ashley worked on the custom shader, and then we worked together on collisions with cubes. In general, we collaborated well because we were able to work through a bulk of the project together in person, which mitigated any miscommuncations, and when we worked asynchronously, we were able to discuss and debug harder problems.