Jekyll2022-03-13T00:04:44+00:00https://vinaybhaip.com/blog/feed.xmlRefinedA deepdive into the thoughts of a young computer scientist.Dissonant Trails2022-03-12T00:00:00+00:002022-03-12T00:00:00+00:00https://vinaybhaip.com/blog/2022/03/12/dissonant-trails<p><img gif="/blog/assets/images/dissonant-trails/three-jump.gif" static="/blog/assets/images/dissonant-trails/three-jump.jpeg" class="gif" width="50%" /></p>
<p><em>An example of a Dissonant Trail. Hover over/tap the image to see how it was constructed. <b>Keep in mind these GIFs are large (~20 MB) if you are on data.</b></em></p>
<h2 id="background">Background</h2>
<p>I’m currently taking a class on Stochastic Processes and thought it’d be cool to make some generative art using what I learned. I have some bigger ideas involving stochastic processes in the works, but in the meanwhile, I thought I’d share some art that I made with it.</p>
<h2 id="making-dissonant-trails">Making Dissonant Trails</h2>
<p>I considered an $n$ discrete state system and wanted to visualize the transitions between the states. Instead of showing the probabilities of the transitions between states, I put particles at the states and simulated multiple timesteps to determine each particle’s path. Visualizing the path gives the “trail” part of the visual. The “dissonant” part comes from the noise of particles colliding with each other.</p>
<p>To make this abstract idea concrete, I used <a href="https://d3js.org">D3</a> to simulate the particles, and used <a href="https://github.com/d3/d3-force">D3-Force</a> to make the particles move between states.</p>
<p>I considered one particle and visualized its path in a system and I got this:</p>
<p><img src="/blog/assets/images/dissonant-trails/process-one.png" width="50%" /></p>
<p>At first, it looks like a bunch of scribbling, but there’s actually some interesting things going on. For example, at the top, the path stayed in the same state but sort-of moved around, probably colliding with other particles. Next, I considered multiple particles’ paths in a system:</p>
<p><img src="/blog/assets/images/dissonant-trails/process-many.png" width="50%" />
<em>Looks like a box of crayons!</em></p>
<p>If you’re interested in how each path is constructed, here’s a video showing the particles over time:</p>
<video width="100%" controls="">
<source src="/blog/assets/images/dissonant-trails/process.mp4" type="video/mp4" />
Video isn't working :(
</video>
<p>At this point, I thought it was pretty cool and started messing with some of the stylistic elements, like the stroke properties and background:</p>
<p><img src="/blog/assets/images/dissonant-trails/process-no-glow.jpeg" width="50%" /></p>
<p><img src="/blog/assets/images/dissonant-trails/process-faded.jpeg" width="50%" /></p>
<p>I wanted to try visualizing even more paths and particles, but the rendering on the page was becoming super slow, so I had to cap it off at a point.</p>
<p>Ultimately, the uniqueness of a generative piece comes from a) the number of states b) the transition matrix c) sampling from the transition matrix and d) moving the states location over time.</p>
<h2 id="examples">Examples</h2>
<!-- <img src="/blog/assets/images/dissonant-trails/left-right.jpeg" width="50%"/>
<img src="/blog/assets/images/dissonant-trails/two-jump.jpeg" width="50%"/>
<img src="/blog/assets/images/dissonant-trails/three-jump.jpeg" width="50%"/>
<img src="/blog/assets/images/dissonant-trails/5-gon.jpeg" width="50%"/>
<img src="/blog/assets/images/dissonant-trails/arc.png" width="50%"/>
<img src="/blog/assets/images/dissonant-trails/circle-20.jpeg" width="50%"/>
-->
<p><img src="/blog/assets/images/dissonant-trails/poster.png" alt="Poster" />
<em>A graphic with some of the dissonant trails</em></p>
<p>Check out the full code to make your own Dissonant Trails <a href="https://github.com/vbhaip/dissonant-trails">here</a>.</p>An example of a Dissonant Trail. Hover over/tap the image to see how it was constructed. Keep in mind these GIFs are large (~20 MB) if you are on data. Background I’m currently taking a class on Stochastic Processes and thought it’d be cool to make some generative art using what I learned. I have some bigger ideas involving stochastic processes in the works, but in the meanwhile, I thought I’d share some art that I made with it. Making Dissonant Trails I considered an $n$ discrete state system and wanted to visualize the transitions between the states. Instead of showing the probabilities of the transitions between states, I put particles at the states and simulated multiple timesteps to determine each particle’s path. Visualizing the path gives the “trail” part of the visual. The “dissonant” part comes from the noise of particles colliding with each other. To make this abstract idea concrete, I used D3 to simulate the particles, and used D3-Force to make the particles move between states. I considered one particle and visualized its path in a system and I got this: At first, it looks like a bunch of scribbling, but there’s actually some interesting things going on. For example, at the top, the path stayed in the same state but sort-of moved around, probably colliding with other particles. Next, I considered multiple particles’ paths in a system: Looks like a box of crayons! If you’re interested in how each path is constructed, here’s a video showing the particles over time: Video isn't working :( At this point, I thought it was pretty cool and started messing with some of the stylistic elements, like the stroke properties and background: I wanted to try visualizing even more paths and particles, but the rendering on the page was becoming super slow, so I had to cap it off at a point. Ultimately, the uniqueness of a generative piece comes from a) the number of states b) the transition matrix c) sampling from the transition matrix and d) moving the states location over time. Examples A graphic with some of the dissonant trails Check out the full code to make your own Dissonant Trails here.Perlin Planets2021-01-26T00:00:00+00:002021-01-26T00:00:00+00:00https://vinaybhaip.com/blog/2021/01/26/perlin-planets<p><img gif="/blog/assets/images/perlin-planets/cyan.gif" static="/blog/assets/images/perlin-planets/cyan.jpg" class="gif" width="50%" />
<em>An example of a Perlin planet. Hover over/tap the image to see it in action. <b>Keep in mind these GIFs are large if you are on data.</b></em></p>
<h2 id="background">Background</h2>
<p>I came across <a href="https://avinayak.github.io/art/2021/01/09/noise-planets.html">this cool blog post</a> from <a href="https://github.com/avinayak">Atul Vinayak</a> on what they call “noise planets.” I recommend checking out the post, but basically they created generative art to replicate art from <a href="https://twitter.com/tylerxhobbs">Tyler Hobbs</a>. The way to generate this is to make flow fields using <a href="https://en.wikipedia.org/wiki/Perlin_noise">Perlin noise</a>, which I found incredibly fascinating. The post was pretty popular on <a href="https://news.ycombinator.com/item?id=25712767">Hacker News</a>, but I noticed one of the comments said it was more like noise circles, rather than noise spheres. I decided to fix that.</p>
<h2 id="making-perlin-planets">Making Perlin Planets</h2>
<p>The first step was replicating the work from the aforementioned blog post. I won’t go into that here because the blog post already does a great job. The main changes I made were making the edges of the circle have less opacity and reducing the stroke width at the edges to give the illusion of depth.</p>
<p><img gif="/blog/assets/images/perlin-planets/flat-planet.gif" static="/blog/assets/images/perlin-planets/flat-planet.png" class="gif" width="50%" />
<em>My first attempt at a Perlin planet. Remember to click it to see the GIF and that it may take a few seconds to load.</em></p>
<p>Looks cool! It was a bit more like a planet, but it wasn’t great. If I really wanted to make it planet-esque, I’d have to take the Perlin noise and put it on an actual sphere.</p>
<p>To do this, I intuitively generated random points on the sphere shell by sampling $\phi \in [0, \pi)$ and $\theta \in [0, 2\pi)$ (<strong>foreshadow: my intuition was wrong</strong>). Then, I used 2D Perlin noise passing in the current location of the point defined by $\phi$ and $\theta$ to determine the next location.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">n</span> <span class="o">=</span> <span class="nx">noise</span><span class="p">(</span><span class="nx">phi</span><span class="p">,</span> <span class="nx">theta</span><span class="p">)</span>
<span class="nx">phi</span> <span class="o">+=</span> <span class="nx">sin</span><span class="p">(</span><span class="mi">2</span><span class="o">*</span><span class="nx">PI</span><span class="o">*</span><span class="nx">n</span><span class="p">);</span>
<span class="nx">theta</span> <span class="o">+=</span> <span class="nx">cos</span><span class="p">(</span><span class="mi">2</span><span class="o">*</span><span class="nx">PI</span><span class="o">*</span><span class="nx">n</span><span class="p">);</span>
</code></pre></div></div>
<p>The last thing to do was to rotate the $\phi$ and $\theta$ of each point to give the feeling of the sphere rotating on an axis.</p>
<p><img gif="/blog/assets/images/perlin-planets/phi-theta-bug.gif" static="/blog/assets/images/perlin-planets/phi-theta-bug.png" class="gif" width="50%" />
<em>This was the result when rotating $\phi$ throughout time.</em></p>
<p>Weird. There’s two issues: 1. everything looks like it’s getting sucked into the center and thrown out on repeat and 2. if you look carefully, the boundary at $\theta = 2\pi$ (or $0$) isn’t continuous.</p>
<p>The first issue is due to a conceptual flaw; if you imagine a ring of points around a sphere’s vertical axis, they all have the same $\phi$. What this means is that incrementing the $\phi$ value is going to make all the points on that ring go up and down the sphere together, which when seen through the top, looks like it’s getting sucked into a vortex.</p>
<p>To deal with this issue, I decided to shift to standard cartesian coordinates. To rotate the coordinates around an axis, I used <a href="https://en.wikipedia.org/wiki/Rodrigues%27_rotation_formula">Rodrigues’ rotation formula</a>.</p>
<p>The second issue has partially to do with the continuity of the flow fields. The idea is that <code class="language-plaintext highlighter-rouge">noise(0)</code> and <code class="language-plaintext highlighter-rouge">noise(2$\pi$)</code> won’t have the same value. This is fixed by saying if $\theta > \pi$, then $\theta = 2\pi - \theta$, for the purposes of drawing the flow fields. This means that it’ll be continuous everywhere.</p>
<p><img gif="/blog/assets/images/perlin-planets/rotating-uneven-distrib.gif" static="/blog/assets/images/perlin-planets/rotating-uneven-distrib.png" class="gif" width="50%" />
<em>Perlin planet with the issues above fixed.</em></p>
<p>A lot better! There’s still one more thing that’s fishy. The points at the poles are much more dense than that of the equator. After a bit of <a href="https://mathworld.wolfram.com/SpherePointPicking.html">research</a>, it turns out that my sampling method of $\phi$ and $\theta$ was wrong! If we think about it, let’s say we have the circle where $\phi = \frac{\pi}{100}$ vs $\phi = \frac{\pi}{2}$. We’d then be sampling $\theta$ from each of these two circles to gives us some points, but we’d have the same number of points on each of the two circles, when the circle at $\phi = \frac{\pi}{100}$ obviously should have less points since it’s a smaller circle. The fix for this is to sample $x, y,$ and $z$ from a Gaussian distribution and normalize them.</p>
<p>The last issue was how long the rendering time for these drawings took. To generate 100 frames, it took roughly 5 minutes. I created a Python script using Selenium to automate the process, adding a query string to specify things like the colors and axis of rotation. To give an even cooler effect, I change the axis of rotation over time as well. The result were some pretty cool Perlin planets.</p>
<h2 id="examples">Examples</h2>
<p><img gif="/blog/assets/images/perlin-planets/pastel-pink.gif" static="/blog/assets/images/perlin-planets/pastel-pink.jpg" class="gif" width="50%" /></p>
<p><img gif="/blog/assets/images/perlin-planets/pastel.gif" static="/blog/assets/images/perlin-planets/pastel.jpg" class="gif" width="50%" /></p>
<p><img gif="/blog/assets/images/perlin-planets/pink-orange.gif" static="/blog/assets/images/perlin-planets/pink-orange.jpg" class="gif" width="50%" /></p>
<p><img src="/blog/assets/images/perlin-planets/poster.png" alt="Poster" />
<em>A graphic with some of the planets</em></p>
<p>Check out the full code to make your own Perlin planets and to see more examples <a href="https://github.com/vbhaip/perlin-planets">here</a>.</p>An example of a Perlin planet. Hover over/tap the image to see it in action. Keep in mind these GIFs are large if you are on data. Background I came across this cool blog post from Atul Vinayak on what they call “noise planets.” I recommend checking out the post, but basically they created generative art to replicate art from Tyler Hobbs. The way to generate this is to make flow fields using Perlin noise, which I found incredibly fascinating. The post was pretty popular on Hacker News, but I noticed one of the comments said it was more like noise circles, rather than noise spheres. I decided to fix that. Making Perlin Planets The first step was replicating the work from the aforementioned blog post. I won’t go into that here because the blog post already does a great job. The main changes I made were making the edges of the circle have less opacity and reducing the stroke width at the edges to give the illusion of depth. My first attempt at a Perlin planet. Remember to click it to see the GIF and that it may take a few seconds to load. Looks cool! It was a bit more like a planet, but it wasn’t great. If I really wanted to make it planet-esque, I’d have to take the Perlin noise and put it on an actual sphere. To do this, I intuitively generated random points on the sphere shell by sampling $\phi \in [0, \pi)$ and $\theta \in [0, 2\pi)$ (foreshadow: my intuition was wrong). Then, I used 2D Perlin noise passing in the current location of the point defined by $\phi$ and $\theta$ to determine the next location. n = noise(phi, theta) phi += sin(2*PI*n); theta += cos(2*PI*n); The last thing to do was to rotate the $\phi$ and $\theta$ of each point to give the feeling of the sphere rotating on an axis. This was the result when rotating $\phi$ throughout time. Weird. There’s two issues: 1. everything looks like it’s getting sucked into the center and thrown out on repeat and 2. if you look carefully, the boundary at $\theta = 2\pi$ (or $0$) isn’t continuous. The first issue is due to a conceptual flaw; if you imagine a ring of points around a sphere’s vertical axis, they all have the same $\phi$. What this means is that incrementing the $\phi$ value is going to make all the points on that ring go up and down the sphere together, which when seen through the top, looks like it’s getting sucked into a vortex. To deal with this issue, I decided to shift to standard cartesian coordinates. To rotate the coordinates around an axis, I used Rodrigues’ rotation formula. The second issue has partially to do with the continuity of the flow fields. The idea is that noise(0) and noise(2$\pi$) won’t have the same value. This is fixed by saying if $\theta > \pi$, then $\theta = 2\pi - \theta$, for the purposes of drawing the flow fields. This means that it’ll be continuous everywhere. Perlin planet with the issues above fixed. A lot better! There’s still one more thing that’s fishy. The points at the poles are much more dense than that of the equator. After a bit of research, it turns out that my sampling method of $\phi$ and $\theta$ was wrong! If we think about it, let’s say we have the circle where $\phi = \frac{\pi}{100}$ vs $\phi = \frac{\pi}{2}$. We’d then be sampling $\theta$ from each of these two circles to gives us some points, but we’d have the same number of points on each of the two circles, when the circle at $\phi = \frac{\pi}{100}$ obviously should have less points since it’s a smaller circle. The fix for this is to sample $x, y,$ and $z$ from a Gaussian distribution and normalize them. The last issue was how long the rendering time for these drawings took. To generate 100 frames, it took roughly 5 minutes. I created a Python script using Selenium to automate the process, adding a query string to specify things like the colors and axis of rotation. To give an even cooler effect, I change the axis of rotation over time as well. The result were some pretty cool Perlin planets. Examples A graphic with some of the planets Check out the full code to make your own Perlin planets and to see more examples here.2020 Favorites2021-01-13T00:00:00+00:002021-01-13T00:00:00+00:00https://vinaybhaip.com/blog/2021/01/13/2020-favorites<p>It’s been a long year, but I’ve grown a lot through autobiographies to personal projects to philosophy papers. In an effort to consolidate some of my favorite things I’ve learned from this year, I put together this short list.</p>
<h3 id="books">Books</h3>
<h5 id="becoming-michelle-obama"><strong>Becoming</strong>, Michelle Obama</h5>
<blockquote>
<p>“For me, becoming isn’t about arriving somewhere or achieving a certain aim. I see it instead as forward motion, a means of evolving, a way to reach continuously toward a better self. The journey doesn’t end.”</p>
</blockquote>
<p>Best book I’ve ever read. The audiobook with her narrating her own story adds even more depth.</p>
<h5 id="when-breath-becomes-air-paul-kalanithi"><strong>When Breath Becomes Air</strong>, Paul Kalanithi</h5>
<blockquote>
<p>“You can’t ever reach perfection, but you can believe in an asymptote toward which you are ceaselessly striving.”</p>
</blockquote>
<p>Technically read last year, but too good to not include. Kalanathi explores his duality as a surgeon and a patient in his exploration for meaning.</p>
<h5 id="what-the-eyes-dont-see-mona-hanna-attisha"><strong>What the Eyes Don’t See</strong>, Mona Hanna-Attisha</h5>
<blockquote>
<p>“This is also a story about the deeper crises we are facing right now in our country: a breakdown in democracy; the disintegration of critical infrastructure dues to inequality and austerity; environmental injustice that disproportionately affects the poor and black; the abandonment of civic responsibility and our deep obligations as human beings to care and provide for one another.”</p>
</blockquote>
<p>A personal perspective on the Flint Water Crisis that makes for a powerful narrative.</p>
<h5 id="educated-tara-westover"><strong>Educated</strong>, Tara Westover</h5>
<h5 id="between-the-world-and-me-ta-nehisi-coates"><strong>Between the World and Me</strong>, Ta-Nehisi Coates</h5>
<h3 id="other-reads">Other Reads</h3>
<h5 id="computing-machinery-and-intelligence-alan-turing"><strong>Computing Machinery and Intelligence</strong>, Alan Turing</h5>
<p>A philosophical examination to answer the question whether machines can think.</p>
<h5 id="simulation-argument-nick-bostrom"><strong><a href="https://www.simulation-argument.com/simulation.html">Simulation Argument</a></strong>, Nick Bostrom</h5>
<h5 id="mangopdfzones-blogpost"><strong><a href="https://mango.pdf.zone/finding-former-australian-prime-minister-tony-abbotts-passport-number-on-instagram">Mango.pdf.zone’s blogpost</a></strong></h5>
<p>Hands-down the best blogpost I’ve ever read.</p>
<h3 id="others">Others</h3>
<h5 id="favorite-video-last-lecture-randy-pausch">Favorite Video: <strong><a href="https://www.youtube.com/watch?v=ji5_MqicxSo">Last Lecture</a></strong>, Randy Pausch</h5>
<p>A meaningful lecture on achieving your childhood dreams (the book is excellent as well).</p>
<h5 id="favorite-project-heliohex">Favorite Project: <strong><a href="https://vinaybhaip.com/blog/2020/07/05/heliohex">HelioHex</a></strong></h5>
<h5 id="favorite-newsletter-hacker-newsletter">Favorite Newsletter: <strong><a href="https://hackernewsletter.com/">Hacker Newsletter</a></strong></h5>
<h5 id="favorite-course-missing-semester">Favorite Course: <strong><a href="https://missing.csail.mit.edu/2020/">Missing Semester</a></strong></h5>It’s been a long year, but I’ve grown a lot through autobiographies to personal projects to philosophy papers. In an effort to consolidate some of my favorite things I’ve learned from this year, I put together this short list. Books Becoming, Michelle Obama “For me, becoming isn’t about arriving somewhere or achieving a certain aim. I see it instead as forward motion, a means of evolving, a way to reach continuously toward a better self. The journey doesn’t end.” Best book I’ve ever read. The audiobook with her narrating her own story adds even more depth. When Breath Becomes Air, Paul Kalanithi “You can’t ever reach perfection, but you can believe in an asymptote toward which you are ceaselessly striving.” Technically read last year, but too good to not include. Kalanathi explores his duality as a surgeon and a patient in his exploration for meaning. What the Eyes Don’t See, Mona Hanna-Attisha “This is also a story about the deeper crises we are facing right now in our country: a breakdown in democracy; the disintegration of critical infrastructure dues to inequality and austerity; environmental injustice that disproportionately affects the poor and black; the abandonment of civic responsibility and our deep obligations as human beings to care and provide for one another.” A personal perspective on the Flint Water Crisis that makes for a powerful narrative. Educated, Tara Westover Between the World and Me, Ta-Nehisi Coates Other Reads Computing Machinery and Intelligence, Alan Turing A philosophical examination to answer the question whether machines can think. Simulation Argument, Nick Bostrom Mango.pdf.zone’s blogpost Hands-down the best blogpost I’ve ever read. Others Favorite Video: Last Lecture, Randy Pausch A meaningful lecture on achieving your childhood dreams (the book is excellent as well). Favorite Project: HelioHex Favorite Newsletter: Hacker Newsletter Favorite Course: Missing SemesterFrom the Sets Rises the Numbers: Constructing the Reals2020-11-22T00:00:00+00:002020-11-22T00:00:00+00:00https://vinaybhaip.com/blog/2020/11/22/sets-to-numbers<script src="https://d3js.org/d3.v6.min.js"></script>
<!--
Part 1: Everything is a Set
- explain set theory and zfc
Part 2: Constructing the reals
-->
<p>What are numbers? In the previous post, we looked at the foundation of set theory to give us a tool to understand this question. Make sure to read that post <a href="https://vinaybhaip.com/blog/2020/11/15/everything-is-a-set">here</a>, and then come back to this article! It’ll be worth it; I promise.</p>
<p>In this post, we’ll dive into the meat of the problem: how do we use sets to define numbers?</p>
<h3 id="from-sets-to-numbers">From Sets to Numbers</h3>
<p>Before we embark on our ambitious task to construct the real numbers, let’s start with something a bit easier. We’ll start with our basic counting numbers, working our way to integers, rational numbers, and then the formiddable foe of the reals. Let’s break down this problem into a couple of levels:</p>
<ol>
<li>$\mathbb{N}$, the natural numbers (0, 1, 2, …)</li>
<li>$\mathbb{Z}$, the integers (…, -2, -1, 0, 1, 2, …)</li>
<li>$\mathbb{Q}$, the rational numbers (fractions)</li>
<li>$\mathbb{R}$, the reals</li>
</ol>
<h4 id="level-1-mathbbn">Level 1: $\mathbb{N}$</h4>
<p>The natural numbers are what we’d consider as basic counting numbers, starting at 0 and going till infinity. (Sidenote: some people believe that natural numbers start at 1, but we’ll start with 0). To beat this level, we need to have a unique representation for all of these numbers.</p>
<p>This seems hard at first; we have to have a unique way to define each number all the way till infinity! Also, we can’t have squirmy definitions, it should be clearly defined since we’re going through all this effort anyways.</p>
<p>Let’s start with the first natural number: $0$. $0$ symbolizes nothing, so it feels natural to assign it as $\emptyset$, or $\{\}$. This is valid from one of our ZFC axioms, saying that there exists an empty set.</p>
<p>Now we have a starting point, how do we construct the next number? Let us define the <strong>successor operation</strong>:</p>
<p>$$S(n) = n \cup \{ n \}$$</p>
<p>This successor function takes a set and returns a new set as above. Let’s do an example: What’s the successor of $\emptyset$?</p>
<p>Well, $S(\emptyset) = \{ \} \cup \{ \{ \} \}$. We’re taking the union of these two sets, so we want to create a set that contains all the elements in each of the two sets. The first set, $\{ \}$, contains no elements, so it won’t “contribute” anything. The second set contains $ \{ \}$, so the union of the two sets must contain it as well. Thus, we get $S(\emptyset) = \{ \{ \} \}$.</p>
<p>Let’s run the successor function again on this result. So we want to calculate $S(S(\emptyset))$.</p>
<p>$$S(S(\emptyset)) = S(\{ \{ \} \}) = S( \{ \emptyset \})$$</p>
<p>$$S( \{ \emptyset \}) = \{ \emptyset \} \cup \{ \{ \emptyset \} \}$$</p>
<p>$$S( \{ \emptyset \}) = \{ \emptyset, \{ \emptyset \} \}$$</p>
<p>That’s a lot of curly braces! Don’t let that distract from what we’re doing: establishing a recursive function that creates a new set from our previous result.</p>
<p>Here’s the cool part: you’ve just defined the natural numbers. We assigned $0$ as $\emptyset$, and then we can assign $1$ as $S(\emptyset)$, $2$ as $S(S(\emptyset))$, and so on.</p>
<p>At this point, I encourage you to try constructing $3$ and check your answer here: <button onclick="this.style.display = 'none'; d3.select('#construct-3').style('display', 'inline'); MathJax.Hub.Queue(['Typeset',MathJax.Hub, 'construct-3']);">Reveal</button> <span id="construct-3" style="display: none"><strong>Answer</strong>: $3 = \{ \emptyset, \{ \emptyset \}, \{\emptyset, \{ \emptyset \} \} \}$</span></p>
<p>Now that we’ve gone through this effort, it’s worth reflecting on what we’ve done. We have defined the successor function to define $n+1$ by calculating $S(n)$. Why do we have to go through this relatively complex process as opposed to other ways to construct $\mathbb{N}$? Let’s look at two candidates for what we could’ve done.</p>
<ol>
<li>
<p>Define $n$ to be the number of items in a set. This set doesn’t have to contain numbers, but just contain random objects. For example, $0 = \{ \}$, $1=\{ red \}$, $2 = \{red, blue\}$. We define a number to be the number of elements in the set.</p>
<p><strong>Problems with this approach</strong>:</p>
<p>a. This is circular! How do we count the number of elements in a set without knowing what numbers are to begin with?</p>
<p>b. There isn’t a <em>unique</em> representation for each number. With this definition, $2 = \{red, blue\}$, but $2 = \{red, green\}$. For every natural number other than $0$, there are an infinite number of ways to represent each number.</p>
</li>
<li>
<p>Define a recursive function such that $n+1 = \{ n\}$. Here, we wrap the previous element in another set and define $0 = \emptyset$.</p>
<p><strong>Problems with this approach</strong>:</p>
<p>Technically, this doesn’t seem to be a bad approach. It simplifies our construction of each number much more. But, there’s a reason we chose to define the natural numbers with the successor function, because it allows for us to understand operations like addition and subtraction much more easily.</p>
<p>There can be many ways to construct the natural numbers; there isn’t one right answer. But we’ll stick to this successor approach.</p>
</li>
</ol>
<p>The last thing to examine is whether this successor approach is really valid. It checks out because we’re using only our axioms for the function: the union operation is defined and the empty set is defined.</p>
<h5 id="defining-operations">Defining operations</h5>
<p>When we talk about natural numbers, we understand what it means to add, multiply and exponentiate. It follows that it should make sense with our construction.</p>
<p>To this end, we can use something called <a href="https://en.wikipedia.org/wiki/Peano_axioms">Peano Arithemetic</a>. To maintain the focus of the article, I won’t go into it here, but if you want to know how we define these important operations, I’d recommend checking out the Wikipedia page!</p>
<h4 id="level-2-mathbbz">Level 2: $\mathbb{Z}$</h4>
<p>For our next challenge, we need to construct the integers. This is like the natural numbers, but includes negative numbers as well.</p>
<p>Let’s think intuitively: how do we know what a negative number is? Well, if I take a number like 0 and subtract 7, then we get -7. But how do we formalize that?</p>
<p>If you recall in the last post, we defined what an ordered pair was. We can write the number -7 as the ordered pair (0, 7), such that 0 - 7 = -7. In this way, we’ve created a negative number using natural numbers (which is totally valid since we’ve defined the natural numbers above).</p>
<p>The problem is that this isn’t unique. If I write the ordered pair $(1,8)$, we get $1-8 = -7$ just like $0-7 = -7$. This shouldn’t be too much of a worry when we define what is known as an <a href="https://en.wikipedia.org/wiki/Equivalence_class">equivalence class</a>. What this means, loosely, is that if I have ordered pairs $(a,b)$ and $(c,d)$ and $a-b = c-d$, then we consider these ordered pairs to exist in the same equivalence class, meaning we can treat them as the same object. If you’d like a more rigorous explanation, I encourage you to check out the Wikipedia page linked above.</p>
<p>We can always also construct the natural numbers this way as well. For example, $3 = (4,1)$. This approach lets us define a value for all the integers.</p>
<h4 id="level-3-mathbbq">Level 3: $\mathbb{Q}$</h4>
<p>Now, we want to construct the rationals, basically just fractions (more explicitly, a quotient between two integers, where the denominator is non-zero). Before you read ahead, I encourage you to think about how we constructed $\mathbb{Z}$ and see if you can come up with a construction for $\mathbb{Q}$.</p>
<p>Hopefully, you recognized we can use ordered pairs again. To represent $\frac{2}{5}$, we can write it as $(2,5)$. To represent $\frac{-17}{14}$, we can write it as $(-17, 14)$. Remember, we can use any of the integers at our disposal because we’ve devised a valid way to construct them.</p>
<p>With the issue that this is not unique if we have a representation like $(2,5)$ and $(4,10)$ representing the same numbers, we can again use the idea from equivalence classes. Two pairs $(a,b)$ and $(c,d)$ are considered to be in the same equivalence class if $\frac{a}{b} = \frac{c}{d}$. If this seems circular to you, we can rewrite it as $ad = bc$ to avoid any problems.</p>
<h4 id="level-4-mathbbr">Level 4: $\mathbb{R}$</h4>
<p>This is what we’ve been waiting for. This is a much harder problem than what we had above: how do we create an all encompassing definition for numbers like $\sqrt{2}$, $\pi$, and $\frac{-e^{17}}{9}$?</p>
<p>Now, we can’t just stick it in an ordered pair like we’ve done before, because we can’t capture all of the reals this way.</p>
<p>Luckily, our good friend Cauchy is here, with what is known as a <strong>Cauchy sequence</strong>. A Cauchy sequence is a sequence (order matters!) of rational numbers such that the sequence converges to some number. We say that this Cauchy sequence is a representation of the number it converges to.</p>
<p>What does convergence mean? Let’s say we have a sequence $\{a_1, a_2, a_3, …\}$. If I were to pick any small positive number $\epsilon$, I should be able to find an $M$ such that all $m > M$ satisfy the following property:</p>
<p>$$ | a_m - L | < \epsilon $$</p>
<p>where $L$ is the limit. More simply, what this says is that the sequence will get closer and closer to the limit value.</p>
<p>Let’s see a visual example of how this works.</p>
<div id="chart2"></div>
<p style="text-align: center;"><span data-name="cauchy-scrub">n=15</span></p>
<script>
//var margin = ({top: 20, right: 120, bottom: 30, left: 120})
//var height = 400, width=700;
var rawwidth = document.getElementsByTagName('article')[0].getBoundingClientRect().width
var margin = ({top: 20, right: rawwidth/10, bottom: 30, left: rawwidth/10})
var height = rawwidth/2, width = rawwidth
var font = 'Georgia'
var temp;
class CauchyGraph {
//data is circle data, items is list of objects to move in the circle
constructor(chart, points, limit){
this.points = points;
this.limit = limit;
this.n = points.length/2;
this.chart = chart;
this.xAxisElement =
this.chart
.append("g")
.attr("id", "yaxis")
.style("font-family", font)
.style("font-size", "2vmin")
.attr('class', 'y axis')
this.yAxisElement =
this.chart
.append("g")
.attr("id", "yaxis")
.style("font-family", font)
.style("font-size", "2vmin")
.attr('class', 'x axis')
}
drawBaseGraph(){
let xScale = d3.scaleLinear([0, this.n], [margin.left, width-margin.right])
let yScale = d3.scaleLinear([Math.min(...this.points)*.9, Math.max(...this.points)*1.1], [height - margin.bottom, margin.top])
console.log(yScale);
let xAxis = d3.axisBottom(xScale).ticks(Math.min(this.n, 20))
let yAxis = d3.axisLeft(yScale)
this.xAxisElement
.attr("transform", `translate(${margin.left},0)`)
.transition()
.call(yAxis);
this.yAxisElement
.attr("transform", `translate(0,${height - margin.bottom})`)
.transition()
.call(xAxis);
var main = this;
function drawDots(){
main.chart
.selectAll(".dot")
.data(main.points.slice(0, main.n))
.enter()
.append("circle")
.attr("class", "dot")
.attr("cx", (d,i) => xScale(i))
.attr("cy", (d) => yScale(d))
.attr("r", 3)
.style("fill", "#232D4B")
.transition();
main.chart
.selectAll(".dot")
.data(main.points.slice(0, main.n))
.exit()
.remove()
main.chart
.selectAll(".dot")
.data(main.points.slice(0, main.n))
.transition()
.attr("cx", (d,i) => xScale(i))
.attr("cy", (d) => yScale(d))
//.append("text")
// .text((d,i) => "(" + i + "," + d + ")")
// .attr("x", (d,i) => xScale(i) + 10)
// .attr("y", (d,i) => yScale(d) + 10)
}
drawDots()
function drawLimit(){
main.chart
.select("#limit")
.remove()
main.chart
.append("line")
.attr("id", "limit")
.attr("x1", xScale(0))
.attr("x2", xScale(main.n))
.attr("y1", yScale(main.limit))
.attr("y2", yScale(main.limit))
.attr("stroke-width", 2)
.attr("stroke-dasharray", ("30,30"))
.attr("stroke", "#E57200")
.transition()
}
drawLimit()
}
makeScrubbableNumber(name, low, high, precision) {
let diagram = this;
let element = d3.select(`[data-name='${name}']`)
.style('color', '#E57200')
.style('font-size', '3vh')
//.style('border', 'dotted 5px #232D4B')
.style('font-weight', 'bold')
.style('text-decoration-line', 'underline')
.style('text-decoration-style', 'dotted')
.style('text-decoration-color', '#232D4B')
.style('text-decoration-skip-ink', 'none')
.style('text-decoration-skip', 'none')
//.style('margin', 'auto')
//.style('width', '100%')
//.style('display', 'block')
.attr('align', 'center')
let positionToValue = d3.scaleLinear()
.clamp(true)
.domain([-400, +400])
.range([low, high]);
element.call(d3.drag()
.on('start', function(event,d){
diagram.startX = positionToValue.invert(diagram.n);
diagram.rawStartX = event.x;
})
.on('drag', function(event, d) {
diagram.n = Math.round(positionToValue(diagram.startX+event.x-diagram.rawStartX));
element.text(() => "n=" + diagram.n);
})
.on('end', function(event, d) {
//console.log(event)
diagram.n = Math.round(positionToValue(diagram.startX+event.x-diagram.rawStartX));
element.text(() => "n=" + diagram.n);
diagram.drawBaseGraph()
//updateNumbers();
}));
// elements.call(d3.drag()
// .on('drag', () => {
// diagram[name] = positionToValue(d3.event.x);
// updateNumbers();
// diagram.update();
// }));
}
}
//https://www.redblobgames.com/making-of/line-drawing/
var chart2 = d3.select('#chart2')
.append('svg')
.attr('width', width)
.attr('height', height)
//var cauchygraph = new CauchyGraph([0, 1, 1.4, 1.41,1.414,1.4142,1.414121], 1.414121);
var cauchygraph = new CauchyGraph(chart2, [0, 10, 2.5, 7.5, 3, 7, 4, 6, 4.5, 5.5, 4.95, 5.05, 4.96, 5.04, 4.97, 5.03, 4.98, 5.02, 4.99, 5.01, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5], 5)
cauchygraph.drawBaseGraph();
cauchygraph.makeScrubbableNumber("cauchy-scrub", 1, cauchygraph.points.length, 0)
//var diagram = new Diagram([data], [{val: "🔵", inset: false}, {val: "plane", inset: false}, {val: "1729", inset: false}])
//document.addEventListener("DOMContentLoaded", function(e) {
// diagram.drawFigure();
// diagram.drawItems();
</script>
<p>Drag the text with $n$ above and see how the graph changes!</p>
<p>What we’re looking at is a sequence of numbers that <em>converges</em> to a limit, in this case $5$. Now, this visualization of the sequence we have above only goes to $n=30$, but we can have an infinite number of elements if we want. Another important idea to consider is that all of these items in the sequence are rational numbers, because we already know how to define them.</p>
<p>The visualization above also only shows the limit of some arbitrary sequence to $5$, but we already know we can define $5$. Let’s try this method with something we know is irrational: $\pi$. Now $\pi$ presents a problem: we can’t represent it as a rational number. We can approximate it with values like $\frac{22}{7}$, but it’ll never <em>be</em> $\pi$.</p>
<p>The way we defined a real number is as a sequence of converging rational numbers. How can we write a sequence for $\pi$? Well, let’s think of this intuitively. We approximate $\pi$ by writing it to a certain number of decimal places. What do I mean by this? $3$ is close to $\pi$, but $3.1$ is even closer, and then $3.14$ is even more closer. We can construct a sequence as the following: $\{3, 3.1, 3.14, 3.141, 3.1415, 3.14159, 3.141592, …\}$. With each item in the sequence, we “reveal” one more number in the decimal.</p>
<p>These decimal numbers can be represented as rational numbers (think about why this is the case!). This means we have our Cauchy sequence! Let’s look at a visualization for how this would look like.</p>
<div id="chart25"></div>
<p style="text-align: center;"><span data-name="cauchy-scrub-2.5">n=50</span></p>
<script>
var chart3 = d3.select('#chart25')
.append('svg')
.attr('width', width)
.attr('height', height)
//var cauchygraph = new CauchyGraph([0, 1, 1.4, 1.41,1.414,1.4142,1.414121], 1.414121);
points = [];
for(let i = 0; i < 100; i++){
let increment = Math.pow(10, i);
points.push(Math.floor(Math.PI*increment)/increment)
}
var cauchygraph = new CauchyGraph(chart3, points, Math.PI)
cauchygraph.drawBaseGraph();
cauchygraph.makeScrubbableNumber("cauchy-scrub-2.5", 1, cauchygraph.points.length, 0)
</script>
<p>A bit anticlimactic. But it looks like a sequence of points that basically approaches the value of $\pi$, which is what we wanted! And these are all rational numbers, so everything checks out.</p>
<p>Let’s try this method with something a bit more obscure: $\frac{\pi^2}{6}$, which is approximately $1.644934$. There’s no way we can write this as a rational, because $\pi$ itself is irrational. So our goal is to find a sequence of rational numbers that converges to this irrational number.</p>
<p>Let’s define a sequence such that each element $a_n = \sum_{x=1}^{n} \frac{1}{x^2}$. According to this definition, the sequence would look like this: $( \frac{1}{1}, \frac{1}{1} + \frac{1}{4}, \frac{1}{1} + \frac{1}{4} + \frac{1}{9}, …)$, more compactly written as $(1, \frac{5}{4}, \frac{49}{36}, …)$.</p>
<p>It’s clear that this satisfies our condition that each element is rational, but does this converge to $\frac{\pi^2}{6}$? Let’s look at our visualization.</p>
<div id="chart3"></div>
<p style="text-align: center;"><span data-name="cauchy-scrub-2">n=50</span></p>
<script>
var chart3 = d3.select('#chart3')
.append('svg')
.attr('width', width)
.attr('height', height)
//var cauchygraph = new CauchyGraph([0, 1, 1.4, 1.41,1.414,1.4142,1.414121], 1.414121);
let points = [1];
for(let i = 1; i < 100; i++){
points.push(points[i-1]+1.0/(i+1)**2)
}
var cauchygraph = new CauchyGraph(chart3, points, 1.644934)
cauchygraph.drawBaseGraph();
cauchygraph.makeScrubbableNumber("cauchy-scrub-2", 1, cauchygraph.points.length, 0)
</script>
<p>It works! We can see that the sequence of rational numbers is converging on $\frac{\pi^2}{6}$. If this sequence kept going on forever, it would converge to this number. This particular sequence was derived from the <a href="https://en.wikipedia.org/wiki/Basel_problem">Basel problem</a>, if you want to look more into it.</p>
<p>We now have an intuitive grasp that it works, but what’s really going on? This post isn’t going to mathematically prove it for you (if you’re curious, you can read up <a href="https://en.wikipedia.org/wiki/Cauchy_sequence">here</a>). The reason Cauchy sequences can work is because the rationals are <a href="https://en.wikipedia.org/wiki/Dense_set">dense</a>. Let’s imagine a unit square, the graph with $x$ within $0$ and $1$ and with $y$ within $0$ and $1$. There are an infinite number of rational numbers in this area. If I were to “zoom in” on any area in this unit square, there would still be an infinite number of rational numbers (I encourage you to think why this is the case).</p>
<p>Now let’s go back to the sequence of rational numbers we have that converge to an irrational number. Because the rational numbers are dense, if I choose any rational number, there is always another rational number that is closer to the irrational number. This is the key reason why we can construct an infinite sequence of rational numbers that converge to an irrational number.</p>
<p>We now have a representation of $\mathbb{R}$! <strong>When we talk about an element in $\mathbb{R}$, such as $\sqrt{2}$, the underlying meaning of $\sqrt{2}$ is an infinite sequence of rational numbers that converges to $\sqrt{2}$.</strong> The last thing to keep in mind is that there can be multiple sequences that converge to $\sqrt{2}$. We can use the same logic that we did in the previous levels, considering sequences that converge to the same number as lying in the same equivalence class.</p>
<p>Now Cauchy sequences are not the only representations of $\mathbb{R}$; another popular method is something called a <a href="https://en.wikipedia.org/wiki/Dedekind_cut">Dedekind cut</a>, which uses a similar construction, but is a little bit different.</p>
<h3 id="conclusion">Conclusion</h3>
<p>And with that, you now know how to construct numbers from purely sets!</p>
<p>Why does this matter? In math, we want to have precise and accurate definitions. Without stable definitions, complex topics become ardent debates about terminology rather than content.</p>
<p>What’s the takeaway? In elementary school, you learned about counting numbers, then fractions, and eventually about real numbers. But what <em>are</em> these numbers? This post showed one way that we can attach a definition to these numbers, more than just abstract understandings of them.</p>
<p>I hope this visual walk-through on constructing the real numbers was helpful. If you think I’ve earned it, I would appreciate subscribing below (I promise there won’t be any spam). If this two-part tutorial was helpful or if you have any feedback, reach out to me at <em>contact at mywebsitename dot com</em>.</p>What are numbers? In the previous post, we looked at the foundation of set theory to give us a tool to understand this question. Make sure to read that post here, and then come back to this article! It’ll be worth it; I promise. In this post, we’ll dive into the meat of the problem: how do we use sets to define numbers? From Sets to Numbers Before we embark on our ambitious task to construct the real numbers, let’s start with something a bit easier. We’ll start with our basic counting numbers, working our way to integers, rational numbers, and then the formiddable foe of the reals. Let’s break down this problem into a couple of levels: $\mathbb{N}$, the natural numbers (0, 1, 2, …) $\mathbb{Z}$, the integers (…, -2, -1, 0, 1, 2, …) $\mathbb{Q}$, the rational numbers (fractions) $\mathbb{R}$, the reals Level 1: $\mathbb{N}$ The natural numbers are what we’d consider as basic counting numbers, starting at 0 and going till infinity. (Sidenote: some people believe that natural numbers start at 1, but we’ll start with 0). To beat this level, we need to have a unique representation for all of these numbers. This seems hard at first; we have to have a unique way to define each number all the way till infinity! Also, we can’t have squirmy definitions, it should be clearly defined since we’re going through all this effort anyways. Let’s start with the first natural number: $0$. $0$ symbolizes nothing, so it feels natural to assign it as $\emptyset$, or $\{\}$. This is valid from one of our ZFC axioms, saying that there exists an empty set. Now we have a starting point, how do we construct the next number? Let us define the successor operation: $$S(n) = n \cup \{ n \}$$ This successor function takes a set and returns a new set as above. Let’s do an example: What’s the successor of $\emptyset$? Well, $S(\emptyset) = \{ \} \cup \{ \{ \} \}$. We’re taking the union of these two sets, so we want to create a set that contains all the elements in each of the two sets. The first set, $\{ \}$, contains no elements, so it won’t “contribute” anything. The second set contains $ \{ \}$, so the union of the two sets must contain it as well. Thus, we get $S(\emptyset) = \{ \{ \} \}$. Let’s run the successor function again on this result. So we want to calculate $S(S(\emptyset))$. $$S(S(\emptyset)) = S(\{ \{ \} \}) = S( \{ \emptyset \})$$ $$S( \{ \emptyset \}) = \{ \emptyset \} \cup \{ \{ \emptyset \} \}$$ $$S( \{ \emptyset \}) = \{ \emptyset, \{ \emptyset \} \}$$ That’s a lot of curly braces! Don’t let that distract from what we’re doing: establishing a recursive function that creates a new set from our previous result. Here’s the cool part: you’ve just defined the natural numbers. We assigned $0$ as $\emptyset$, and then we can assign $1$ as $S(\emptyset)$, $2$ as $S(S(\emptyset))$, and so on. At this point, I encourage you to try constructing $3$ and check your answer here: Reveal Answer: $3 = \{ \emptyset, \{ \emptyset \}, \{\emptyset, \{ \emptyset \} \} \}$ Now that we’ve gone through this effort, it’s worth reflecting on what we’ve done. We have defined the successor function to define $n+1$ by calculating $S(n)$. Why do we have to go through this relatively complex process as opposed to other ways to construct $\mathbb{N}$? Let’s look at two candidates for what we could’ve done. Define $n$ to be the number of items in a set. This set doesn’t have to contain numbers, but just contain random objects. For example, $0 = \{ \}$, $1=\{ red \}$, $2 = \{red, blue\}$. We define a number to be the number of elements in the set. Problems with this approach: a. This is circular! How do we count the number of elements in a set without knowing what numbers are to begin with? b. There isn’t a unique representation for each number. With this definition, $2 = \{red, blue\}$, but $2 = \{red, green\}$. For every natural number other than $0$, there are an infinite number of ways to represent each number. Define a recursive function such that $n+1 = \{ n\}$. Here, we wrap the previous element in another set and define $0 = \emptyset$. Problems with this approach: Technically, this doesn’t seem to be a bad approach. It simplifies our construction of each number much more. But, there’s a reason we chose to define the natural numbers with the successor function, because it allows for us to understand operations like addition and subtraction much more easily. There can be many ways to construct the natural numbers; there isn’t one right answer. But we’ll stick to this successor approach. The last thing to examine is whether this successor approach is really valid. It checks out because we’re using only our axioms for the function: the union operation is defined and the empty set is defined. Defining operations When we talk about natural numbers, we understand what it means to add, multiply and exponentiate. It follows that it should make sense with our construction. To this end, we can use something called Peano Arithemetic. To maintain the focus of the article, I won’t go into it here, but if you want to know how we define these important operations, I’d recommend checking out the Wikipedia page! Level 2: $\mathbb{Z}$ For our next challenge, we need to construct the integers. This is like the natural numbers, but includes negative numbers as well. Let’s think intuitively: how do we know what a negative number is? Well, if I take a number like 0 and subtract 7, then we get -7. But how do we formalize that? If you recall in the last post, we defined what an ordered pair was. We can write the number -7 as the ordered pair (0, 7), such that 0 - 7 = -7. In this way, we’ve created a negative number using natural numbers (which is totally valid since we’ve defined the natural numbers above). The problem is that this isn’t unique. If I write the ordered pair $(1,8)$, we get $1-8 = -7$ just like $0-7 = -7$. This shouldn’t be too much of a worry when we define what is known as an equivalence class. What this means, loosely, is that if I have ordered pairs $(a,b)$ and $(c,d)$ and $a-b = c-d$, then we consider these ordered pairs to exist in the same equivalence class, meaning we can treat them as the same object. If you’d like a more rigorous explanation, I encourage you to check out the Wikipedia page linked above. We can always also construct the natural numbers this way as well. For example, $3 = (4,1)$. This approach lets us define a value for all the integers. Level 3: $\mathbb{Q}$ Now, we want to construct the rationals, basically just fractions (more explicitly, a quotient between two integers, where the denominator is non-zero). Before you read ahead, I encourage you to think about how we constructed $\mathbb{Z}$ and see if you can come up with a construction for $\mathbb{Q}$. Hopefully, you recognized we can use ordered pairs again. To represent $\frac{2}{5}$, we can write it as $(2,5)$. To represent $\frac{-17}{14}$, we can write it as $(-17, 14)$. Remember, we can use any of the integers at our disposal because we’ve devised a valid way to construct them. With the issue that this is not unique if we have a representation like $(2,5)$ and $(4,10)$ representing the same numbers, we can again use the idea from equivalence classes. Two pairs $(a,b)$ and $(c,d)$ are considered to be in the same equivalence class if $\frac{a}{b} = \frac{c}{d}$. If this seems circular to you, we can rewrite it as $ad = bc$ to avoid any problems. Level 4: $\mathbb{R}$ This is what we’ve been waiting for. This is a much harder problem than what we had above: how do we create an all encompassing definition for numbers like $\sqrt{2}$, $\pi$, and $\frac{-e^{17}}{9}$? Now, we can’t just stick it in an ordered pair like we’ve done before, because we can’t capture all of the reals this way. Luckily, our good friend Cauchy is here, with what is known as a Cauchy sequence. A Cauchy sequence is a sequence (order matters!) of rational numbers such that the sequence converges to some number. We say that this Cauchy sequence is a representation of the number it converges to. What does convergence mean? Let’s say we have a sequence $\{a_1, a_2, a_3, …\}$. If I were to pick any small positive number $\epsilon$, I should be able to find an $M$ such that all $m > M$ satisfy the following property: $$ | a_m - L | < \epsilon $$ where $L$ is the limit. More simply, what this says is that the sequence will get closer and closer to the limit value. Let’s see a visual example of how this works. n=15 Drag the text with $n$ above and see how the graph changes! What we’re looking at is a sequence of numbers that converges to a limit, in this case $5$. Now, this visualization of the sequence we have above only goes to $n=30$, but we can have an infinite number of elements if we want. Another important idea to consider is that all of these items in the sequence are rational numbers, because we already know how to define them. The visualization above also only shows the limit of some arbitrary sequence to $5$, but we already know we can define $5$. Let’s try this method with something we know is irrational: $\pi$. Now $\pi$ presents a problem: we can’t represent it as a rational number. We can approximate it with values like $\frac{22}{7}$, but it’ll never be $\pi$. The way we defined a real number is as a sequence of converging rational numbers. How can we write a sequence for $\pi$? Well, let’s think of this intuitively. We approximate $\pi$ by writing it to a certain number of decimal places. What do I mean by this? $3$ is close to $\pi$, but $3.1$ is even closer, and then $3.14$ is even more closer. We can construct a sequence as the following: $\{3, 3.1, 3.14, 3.141, 3.1415, 3.14159, 3.141592, …\}$. With each item in the sequence, we “reveal” one more number in the decimal. These decimal numbers can be represented as rational numbers (think about why this is the case!). This means we have our Cauchy sequence! Let’s look at a visualization for how this would look like. n=50 A bit anticlimactic. But it looks like a sequence of points that basically approaches the value of $\pi$, which is what we wanted! And these are all rational numbers, so everything checks out. Let’s try this method with something a bit more obscure: $\frac{\pi^2}{6}$, which is approximately $1.644934$. There’s no way we can write this as a rational, because $\pi$ itself is irrational. So our goal is to find a sequence of rational numbers that converges to this irrational number. Let’s define a sequence such that each element $a_n = \sum_{x=1}^{n} \frac{1}{x^2}$. According to this definition, the sequence would look like this: $( \frac{1}{1}, \frac{1}{1} + \frac{1}{4}, \frac{1}{1} + \frac{1}{4} + \frac{1}{9}, …)$, more compactly written as $(1, \frac{5}{4}, \frac{49}{36}, …)$. It’s clear that this satisfies our condition that each element is rational, but does this converge to $\frac{\pi^2}{6}$? Let’s look at our visualization. n=50 It works! We can see that the sequence of rational numbers is converging on $\frac{\pi^2}{6}$. If this sequence kept going on forever, it would converge to this number. This particular sequence was derived from the Basel problem, if you want to look more into it. We now have an intuitive grasp that it works, but what’s really going on? This post isn’t going to mathematically prove it for you (if you’re curious, you can read up here). The reason Cauchy sequences can work is because the rationals are dense. Let’s imagine a unit square, the graph with $x$ within $0$ and $1$ and with $y$ within $0$ and $1$. There are an infinite number of rational numbers in this area. If I were to “zoom in” on any area in this unit square, there would still be an infinite number of rational numbers (I encourage you to think why this is the case). Now let’s go back to the sequence of rational numbers we have that converge to an irrational number. Because the rational numbers are dense, if I choose any rational number, there is always another rational number that is closer to the irrational number. This is the key reason why we can construct an infinite sequence of rational numbers that converge to an irrational number. We now have a representation of $\mathbb{R}$! When we talk about an element in $\mathbb{R}$, such as $\sqrt{2}$, the underlying meaning of $\sqrt{2}$ is an infinite sequence of rational numbers that converges to $\sqrt{2}$. The last thing to keep in mind is that there can be multiple sequences that converge to $\sqrt{2}$. We can use the same logic that we did in the previous levels, considering sequences that converge to the same number as lying in the same equivalence class. Now Cauchy sequences are not the only representations of $\mathbb{R}$; another popular method is something called a Dedekind cut, which uses a similar construction, but is a little bit different. Conclusion And with that, you now know how to construct numbers from purely sets! Why does this matter? In math, we want to have precise and accurate definitions. Without stable definitions, complex topics become ardent debates about terminology rather than content. What’s the takeaway? In elementary school, you learned about counting numbers, then fractions, and eventually about real numbers. But what are these numbers? This post showed one way that we can attach a definition to these numbers, more than just abstract understandings of them. I hope this visual walk-through on constructing the real numbers was helpful. If you think I’ve earned it, I would appreciate subscribing below (I promise there won’t be any spam). If this two-part tutorial was helpful or if you have any feedback, reach out to me at contact at mywebsitename dot com.Everything is a Set: Constructing the Reals2020-11-15T00:00:00+00:002020-11-15T00:00:00+00:00https://vinaybhaip.com/blog/2020/11/15/everything-is-a-set<script src="https://d3js.org/d3.v6.min.js"></script>
<!--
Part 1: Everything is a Set
- explain set theory and zfc
Part 2: Constructing the reals
-->
<p>What does it mean when we say $\sqrt{2}$? Well, you may say, it’s when we have a number $x$ such that $x^2 = 2$. That may satisfy you, but it shouldn’t. The notion of this abstract $\sqrt{2}$ is weird in that we define it as the “opposite” of squaring two. What does $\sqrt{2}$ <em>look</em> like?</p>
<p>Let’s take a step back. What do we mean when we say $2$? I can already hear the arguments, “Vinay, it’s two. Come on. I have two eyes. I have two ears. It’s easy.” To this end, we have a more concrete idea of what $2$ <em>looks</em> like, but we still don’t know what it is. This may seem acceptable to you, but considering that math is built on numbers, it’s worthwhile to examine what we mean by numbers. Here, I will give a walkthrough on constructing the real numbers, $\mathbb{R}$, which comprises of numbers like: $0, 42, -7, \frac{9}{17},$ and $\sqrt{38.9}$ to list a few.</p>
<p>By the end of this series on constructing the real numbers, you’ll learn how to represent a real number using foundational mathematical objects.</p>
<h3 id="what-is-a-set">What is a set?</h3>
<p><strong>Note, feel free to skip this section if you know what sets are.</strong></p>
<p>Since we’re taking on the task of constructing the real numbers, we need some tool. In this case, we’ll be looking at set theory as the basis for understanding objects.</p>
<p>We will use sets. There’s only one thing that a set tells us: whether an object is in the set or not. That’s it.</p>
<p>What are the implications of this definition? Objects in sets are unique. Second, there is no order to a set. We only know whether an object is in a set or not, meaning it has no position.</p>
<p>Let’s look at an interactive example. Move the items on the right in and out of the circle to see how the set looks like.</p>
<h3 id="a--"><span id="d3-1">$$A = \{\}$$</span></h3>
<div id="chart"></div>
<script>
var margin = ({top: 20, right: 120, bottom: 30, left: 120})
var height = 400, width=700;
var font = 'Georgia'
var data = {x: 50, y: 50, r: 25}
var temp;
class Diagram {
//data is circle data, items is list of objects to move in the circle
constructor(data, items){
this.data = data;
this.items = items;
temp = this;
}
drawSet(){
// svg
// .append("text")
// .text((d) => "$\{hi, yes, no\}$")
// .attr("x", width/2)
// .attr("y", 3*height/4)
//$\$A = \\{\\}\$$
let toWrite = "$\$A = \\{ "
for(let i = 0; i < this.items.length; i++){
if(this.items[i].inset){
toWrite += (this.items[i].val + ",");
}
}
toWrite = toWrite.substring(0,toWrite.length-1);
toWrite += "\\}\$$"
d3.select("#d3-1")
.text(toWrite)
}
drag = d3.drag()
.on("drag", function(event, d) {
d3.select(this)
.attr("x", event.x)
.attr("y", event.y)
function distInThresh(x1, y1, x2, y2, thresh) {
return (x1-x2)**2 + (y1-y2)**2 < thresh**2;
}
let set = d3.select("#set")
let before = d.inset;
if(distInThresh(event.x, event.y, set.attr("cx"), set.attr("cy"), set.attr("r"))){
d.inset = true;
}
else{
d.inset = false;
}
if(d.inset !== before){
temp.drawSet();
MathJax.Hub.Queue(["Typeset",MathJax.Hub]);
}
})
drawFigure() {
svg
.data(this.data)
.append("circle")
.attr("id", "set")
.style("stroke", "orange")
.style("stroke-width", 5)
.style("fill", "none")
.attr("r", width/5)
.attr("cx", width/4)
.attr("cy", height/2)
}
drawItems() {
svg
.selectAll("text")
.data(this.items)
//.enter()
.join("text")
.text((d) => d.val)
.attr("x", (d,i) => width/2 + width/10*(i+1))
.attr("y", height/2)
.style("font-size", 20)
.style("cursor", "pointer")
.call(this.drag)
}
}
var data = {x: 50, y: 50, r: 25}
var svg = d3.select('#chart')
.append('svg')
.attr('width', width)
.attr('height', height)
var diagram = new Diagram([data], [{val: "🔵", inset: false}, {val: "plane", inset: false}, {val: "1729", inset: false}])
document.addEventListener("DOMContentLoaded", function(e) {
diagram.drawFigure();
diagram.drawItems();
});
</script>
<p>Let’s do a quick review before we move on.</p>
<ol>
<li>
<p>Is $\{marshmallow, red, 📘\}$ a set?<button onclick="this.style.display = 'none'; d3.select('#q1').style('display', 'inline');">Reveal</button> <span id="q1" style="display: none"><strong>Yes</strong>. Sets can have whatever we want in them.</span></p>
</li>
<li>
<p>If I have $\{x, orange, laptop\}$, and I added the element “x” to it, what does the set look like? <button onclick="this.style.display = 'none'; d3.select('#q2').style('display', 'inline');">Reveal</button> <span id="q2" style="display: none"> <strong>Nothing changes</strong>. Sets can’t have duplicates. As a sidenote, if you do find a set that is written to have repeating elements, then that set doesn’t make sense; think of it like a typo in a sentence.</span></p>
</li>
<li>
<p>Are $\{falcon, blue\}$ and $\{blue, falcon\}$ different? <button onclick="this.style.display = 'none'; d3.select('#q3').style('display', 'inline');">Reveal</button> <span id="q3" style="display: none"> <strong>Nope</strong>. Don’t let the order of elements trick you, these two are the same sets.</span></p>
</li>
</ol>
<p>We now have the basics of sets down! Now to get to the interesting stuff.</p>
<h3 id="zfc">ZFC</h3>
<p>Now that we know what a set is, we need to have some basic truths we can agree on. These are known as <strong>axioms</strong>. I can’t prove these to you, but think of them as definitions. This is the foundation for everything we will do.</p>
<p>The way we’ll do this is through <a href="https://en.wikipedia.org/wiki/Zermelo%E2%80%93Fraenkel_set_theory">Zermelo–Fraenkel set theory</a>. There is so much to look at here, but we’ll stick to the most relevant aspects of the theory.</p>
<ol>
<li>
<p><strong>There exists a set.</strong> I think we can agree on this one. This just says that it is possible to construct a set that obeys the rules like above.</p>
</li>
<li>
<p><strong>There exists an empty set.</strong> An empty set is one which has no elements. This should feel like a reasonable assumption. Note, this set is unique. There’s only one way to exclude every object, and that’s by not having any object in it. We denote this set as $\emptyset$, which is just some mathematical notation; nothing to fear.</p>
</li>
<li>
<p><strong>We can create a set that contains all the elements of two sets.</strong> This is the union operation. It’s best to show this by example: $A=\{hi, blue\}$ and $B=\{green\}$. Then $A \cup B = \{hi, blue, green\}$.</p>
</li>
</ol>
<p>There are other axioms in ZFC, but these are the bare necessity for what we need to do.</p>
<h3 id="ordered-pairs-using-sets">Ordered Pairs Using Sets</h3>
<p>This will be helpful later, but let’s try to understand how we’d represent an ordered pair using sets. Why do we care about this? Well, it seems reasonable we’d want to represent an object $(a, b)$ to be different than $(b, a)$. Essentially, we want to construct an ordered pair using an unordered set.</p>
<p>How does this make sense?! How do we create order from no order? We can’t just throw them in a set because $\{a, b\}$ is the same as $\{b, a\}$.</p>
<p>The solution is to encode each item as a set itself that contains previous elements. Let’s look at an example to understand this.</p>
<p>We want to represent $(a,b)$ as a set. According to what I said above, we would write this as a set like $\{ \{a\}, \{a,b\} \}$. How does this fix our problem? Let’s try to represent $(b,a)$: we get $\{ \{b\}, \{b,a\} \}$. Both sets have the item $\{a,b\}$ in it, but the difference is the set with either only $a$ or only $b$.</p>
<p>Now something important to see here is that inside each set, we can indeed have another set, because it’s just another element.</p>
<p>This way of constructing ordered pairs is meant to provide you with a reason to believe that we can do so. If this concept feels weird, don’t stress. The main takeaway is that ordered pairs can exist from sets. This allows us to construct something called a <strong>sequence</strong>, a collection of objects where the objects are ordered. A sequence is an “extended” ordered pair: it can contain as many elements as we’d like and maintain an order.</p>
<h3 id="wrapping-up">Wrapping Up</h3>
<p>This post should’ve given you a brief background on set theory, the goal of which is to construct mathematical objects from this one basic concept: a set. We talked about some of the basic ZFC axioms and introduced how we can construct an ordered collection from this unordered collection.</p>
<p>Now what? We have the foundation for constructing the reals. The next post will do a deepdive in constructing the real numbers, starting with your basic counting numbers like $0,1,2,$ and so forth. We’re going to construct these complicated objects solely from the information above. As a plus, it’ll use cool interactive graphs to explain the concepts!</p>
<p>If this interests you (which it should!), consider subscribing to get updates on when the next post drops. Until then, I encourage you to think about how you’d go about constructing numbers using only the information we’ve covered!</p>
<p><strong>Update: Check out the follow up post <a href="https://vinaybhaip.com/blog/2020/11/22/sets-to-numbers">here</a>!</strong></p>What does it mean when we say $\sqrt{2}$? Well, you may say, it’s when we have a number $x$ such that $x^2 = 2$. That may satisfy you, but it shouldn’t. The notion of this abstract $\sqrt{2}$ is weird in that we define it as the “opposite” of squaring two. What does $\sqrt{2}$ look like? Let’s take a step back. What do we mean when we say $2$? I can already hear the arguments, “Vinay, it’s two. Come on. I have two eyes. I have two ears. It’s easy.” To this end, we have a more concrete idea of what $2$ looks like, but we still don’t know what it is. This may seem acceptable to you, but considering that math is built on numbers, it’s worthwhile to examine what we mean by numbers. Here, I will give a walkthrough on constructing the real numbers, $\mathbb{R}$, which comprises of numbers like: $0, 42, -7, \frac{9}{17},$ and $\sqrt{38.9}$ to list a few. By the end of this series on constructing the real numbers, you’ll learn how to represent a real number using foundational mathematical objects. What is a set? Note, feel free to skip this section if you know what sets are. Since we’re taking on the task of constructing the real numbers, we need some tool. In this case, we’ll be looking at set theory as the basis for understanding objects. We will use sets. There’s only one thing that a set tells us: whether an object is in the set or not. That’s it. What are the implications of this definition? Objects in sets are unique. Second, there is no order to a set. We only know whether an object is in a set or not, meaning it has no position. Let’s look at an interactive example. Move the items on the right in and out of the circle to see how the set looks like. $$A = \{\}$$ Let’s do a quick review before we move on. Is $\{marshmallow, red, 📘\}$ a set?Reveal Yes. Sets can have whatever we want in them. If I have $\{x, orange, laptop\}$, and I added the element “x” to it, what does the set look like? Reveal Nothing changes. Sets can’t have duplicates. As a sidenote, if you do find a set that is written to have repeating elements, then that set doesn’t make sense; think of it like a typo in a sentence. Are $\{falcon, blue\}$ and $\{blue, falcon\}$ different? Reveal Nope. Don’t let the order of elements trick you, these two are the same sets. We now have the basics of sets down! Now to get to the interesting stuff. ZFC Now that we know what a set is, we need to have some basic truths we can agree on. These are known as axioms. I can’t prove these to you, but think of them as definitions. This is the foundation for everything we will do. The way we’ll do this is through Zermelo–Fraenkel set theory. There is so much to look at here, but we’ll stick to the most relevant aspects of the theory. There exists a set. I think we can agree on this one. This just says that it is possible to construct a set that obeys the rules like above. There exists an empty set. An empty set is one which has no elements. This should feel like a reasonable assumption. Note, this set is unique. There’s only one way to exclude every object, and that’s by not having any object in it. We denote this set as $\emptyset$, which is just some mathematical notation; nothing to fear. We can create a set that contains all the elements of two sets. This is the union operation. It’s best to show this by example: $A=\{hi, blue\}$ and $B=\{green\}$. Then $A \cup B = \{hi, blue, green\}$. There are other axioms in ZFC, but these are the bare necessity for what we need to do. Ordered Pairs Using Sets This will be helpful later, but let’s try to understand how we’d represent an ordered pair using sets. Why do we care about this? Well, it seems reasonable we’d want to represent an object $(a, b)$ to be different than $(b, a)$. Essentially, we want to construct an ordered pair using an unordered set. How does this make sense?! How do we create order from no order? We can’t just throw them in a set because $\{a, b\}$ is the same as $\{b, a\}$. The solution is to encode each item as a set itself that contains previous elements. Let’s look at an example to understand this. We want to represent $(a,b)$ as a set. According to what I said above, we would write this as a set like $\{ \{a\}, \{a,b\} \}$. How does this fix our problem? Let’s try to represent $(b,a)$: we get $\{ \{b\}, \{b,a\} \}$. Both sets have the item $\{a,b\}$ in it, but the difference is the set with either only $a$ or only $b$. Now something important to see here is that inside each set, we can indeed have another set, because it’s just another element. This way of constructing ordered pairs is meant to provide you with a reason to believe that we can do so. If this concept feels weird, don’t stress. The main takeaway is that ordered pairs can exist from sets. This allows us to construct something called a sequence, a collection of objects where the objects are ordered. A sequence is an “extended” ordered pair: it can contain as many elements as we’d like and maintain an order. Wrapping Up This post should’ve given you a brief background on set theory, the goal of which is to construct mathematical objects from this one basic concept: a set. We talked about some of the basic ZFC axioms and introduced how we can construct an ordered collection from this unordered collection. Now what? We have the foundation for constructing the reals. The next post will do a deepdive in constructing the real numbers, starting with your basic counting numbers like $0,1,2,$ and so forth. We’re going to construct these complicated objects solely from the information above. As a plus, it’ll use cool interactive graphs to explain the concepts! If this interests you (which it should!), consider subscribing to get updates on when the next post drops. Until then, I encourage you to think about how you’d go about constructing numbers using only the information we’ve covered! Update: Check out the follow up post here!Summer of Stories2020-10-24T00:00:00+00:002020-10-24T00:00:00+00:00https://vinaybhaip.com/blog/2020/10/24/summer-of-stories<p>My summer started in March, when school basically finished. Grappled with what I’ve always desired, unlimited free time, I sought a productive outlet. In this post, I’ll outline some of the highlights of my summer.</p>
<p>I started my break with a goal to refine <em>how</em> I code with the <a href="https://missing.csail.mit.edu/2020/">Missing Semester MIT course</a>. This course goes into workflow tools for coding, ranging from teaching Bash to explaining cryptography. It was through this course that I discovered my love for Vim. It no longer was this abstract editor of hackers; it was a way to “code” the way I code. And I can’t explain how much time I’ve saved using Vim. The only problem is the time I’ve saved using Vim, I spend to convince others to use Vim. But it doesn’t matter; I just feel like a superhero using Vim.</p>
<p>Back to the topic, from this course I also learned about Git. Now, I already knew how to use Git, but this course showed me the inner workings, going into the ideas of trees, blobs, and directed acyclic graphs. What this is to say is that I became better at Git, and it was valuable.</p>
<p><img src="https://imgs.xkcd.com/comics/git.png" alt="" />
<em>As always, a relevant comic from <a href="https://www.xkcd.com">xkcd</a></em></p>
<p>The next milestone of my summer was rebuilding my personal website, something I’m really proud about. A more in-depth explanation of how I did this can be found <a href="https://vinaybhaip.com/blog/2020/07/18/website-overhaul">here</a>. In short, I modernized the website. It had been about 2.5 years since I originally made my website, but my new design was <em>vastly</em> different from the original one. Looking back, it’s cool seeing just how much I’ve grown in a relatively short amount of time.</p>
<p>The highlight of my summer was by far making HelioHex: a highly-configurable, modular, music-syncing lighting system. I wrote a post which has a demo and a tutorial of how I made it <a href="http://127.0.0.1:4000/blog/2020/10/16/summer-recap">here</a>. I don’t think I could do it justice with a summary here, so I <strong>highly</strong> recommend checking it out.</p>
<p>The bulk of my summer was spent working with the <a href="https://www.janelia.org/lab/looger-lab">Looger Lab</a> along with the <a href="https://www.janelia.org/lab/turaga-lab">Turaga Lab</a> to use machine learning to devise better calcium indicators of neural activity. I worked with some amazing people across disciplines from chemistry to neuroscience to computer science. My summer work was amazing; from devising and constructing models using TensorFlow to writing regression tests, I felt like a real machine learning engineer. The coolest part was applying all my newfound knowledge from the Missing Semester course in scripting and Git.</p>
<p>With roughly a week left before school started, my family encouraged me to enjoy my remaining time before the stress of school hits. The problem is that I find it hard to kick back for extended periods of time. Don’t get me wrong, I love to relax, but I enjoy myself more when I’m productive. So after taking a few days off, I spent the last half-week of summer vacation learning D3.js and creating a <a href="https://vinaybhaip.com/blog/2020/08/22/spotify-artist-viz">visualization of my Spotify listening patterns</a>. What this project taught me is that I <strong>love</strong> D3.js. It’s a bit complicated to understand the basics, but now that I do, I feel the world at my fingertips. I want to visualize data because I have a drive to tell stories. And so a few days on what was supposed to be a side-project now feels like a legitimate career path. Funny how things work like that 🙂.</p>
<h3 id="outside-of-cs">Outside of CS</h3>
<p>Outside of refining my computer science skills, I learned how to draw better. I followed the <a href="https://drawabox.com/">DrawABox</a> lessons, which at one point, comprised of drawing 200 boxes freehand. It took a lot of work, but definitely helped me understand constructions better. I took a step back once school started, but hopefully I can dive back in during winter break.</p>
<p>Also, I’ve spent some time enjoying nature through walks and runs. Often, my mind needs space to escape the confines of the blue glow from my laptop screen, and taking in the scenery outside can help. It amazes me just how helpful it is, but it’s hard to build this as a habit. It’ll be something I’ll be working on in the future.</p>
<p>Lastly, I’ve read some pretty great books. I’m saving them for a future post, so stay on the lookout! Most of the books I read were personal biographies, a genre that I resonate with. There’s something comforting in listening to human stories, listening to how people learn, grow, and make the world around them better.</p>
<p>To this end, I’m calling this summer the Summer of Stories. From making a scrollyteller of my Spotify listening patterns to listening to autobiographies to witnessing lives of people affected by the pandemic to telling my own narrative with this blog, this summer feels like it revolved around stories. I’ve learned that I love stories because they make me feel human. They make me feel like I can connect to the experiences of others.</p>
<p>In summer-ry (get the joke?), I’d say this summer was pretty productive. I’ve picked up some cool new computer science tools and learned from valuable experiences. The optimist in me finds this reassuring that even in a pandemic, I can grow as a person.</p>
<!--
- bash, felt more comfortable. i "sorta" knew what i was doing, but this made me a lot more strong in scripting
- git, i knew git before for sure. but i didn't know how it worked underlying
- xkcd relevant https://xkcd.com/1597/
- trees, blobs, DAGs
- first got bash skills on lock
- mit missing semester
- rebuilt my website - much needed
- check out the post https://vinaybhaip.com/blog/2020/07/18/website-overhaul
- also added the blog you're reading!
- modernized
- heliohex
- check it out!
- not gonna go much into it, but this is what is the focal point of my summer
- you have to check out the blogpost for this!
- worked over the summer @ looger lab
- lucky to work with amazing people on designing proteins
- used shell scripting from missing semester course, which was awesome
- built off ml stuff from jayaraman lab
- last days of summer worked on d3
- check out spotify post
- something else in the works, but it's gone on the burner
- hopefully will get time for that later
- lastly, learned about taking breaks
- stepping back, going running and walking outside
- taking it all in is just really great
- refreshing break from coding all day
- worked on drawabox
- read some great books, including Educated by Tara Westover
- reading list probably coming out w/ fav books
-->My summer started in March, when school basically finished. Grappled with what I’ve always desired, unlimited free time, I sought a productive outlet. In this post, I’ll outline some of the highlights of my summer. I started my break with a goal to refine how I code with the Missing Semester MIT course. This course goes into workflow tools for coding, ranging from teaching Bash to explaining cryptography. It was through this course that I discovered my love for Vim. It no longer was this abstract editor of hackers; it was a way to “code” the way I code. And I can’t explain how much time I’ve saved using Vim. The only problem is the time I’ve saved using Vim, I spend to convince others to use Vim. But it doesn’t matter; I just feel like a superhero using Vim. Back to the topic, from this course I also learned about Git. Now, I already knew how to use Git, but this course showed me the inner workings, going into the ideas of trees, blobs, and directed acyclic graphs. What this is to say is that I became better at Git, and it was valuable. As always, a relevant comic from xkcd The next milestone of my summer was rebuilding my personal website, something I’m really proud about. A more in-depth explanation of how I did this can be found here. In short, I modernized the website. It had been about 2.5 years since I originally made my website, but my new design was vastly different from the original one. Looking back, it’s cool seeing just how much I’ve grown in a relatively short amount of time. The highlight of my summer was by far making HelioHex: a highly-configurable, modular, music-syncing lighting system. I wrote a post which has a demo and a tutorial of how I made it here. I don’t think I could do it justice with a summary here, so I highly recommend checking it out. The bulk of my summer was spent working with the Looger Lab along with the Turaga Lab to use machine learning to devise better calcium indicators of neural activity. I worked with some amazing people across disciplines from chemistry to neuroscience to computer science. My summer work was amazing; from devising and constructing models using TensorFlow to writing regression tests, I felt like a real machine learning engineer. The coolest part was applying all my newfound knowledge from the Missing Semester course in scripting and Git. With roughly a week left before school started, my family encouraged me to enjoy my remaining time before the stress of school hits. The problem is that I find it hard to kick back for extended periods of time. Don’t get me wrong, I love to relax, but I enjoy myself more when I’m productive. So after taking a few days off, I spent the last half-week of summer vacation learning D3.js and creating a visualization of my Spotify listening patterns. What this project taught me is that I love D3.js. It’s a bit complicated to understand the basics, but now that I do, I feel the world at my fingertips. I want to visualize data because I have a drive to tell stories. And so a few days on what was supposed to be a side-project now feels like a legitimate career path. Funny how things work like that 🙂. Outside of CS Outside of refining my computer science skills, I learned how to draw better. I followed the DrawABox lessons, which at one point, comprised of drawing 200 boxes freehand. It took a lot of work, but definitely helped me understand constructions better. I took a step back once school started, but hopefully I can dive back in during winter break. Also, I’ve spent some time enjoying nature through walks and runs. Often, my mind needs space to escape the confines of the blue glow from my laptop screen, and taking in the scenery outside can help. It amazes me just how helpful it is, but it’s hard to build this as a habit. It’ll be something I’ll be working on in the future. Lastly, I’ve read some pretty great books. I’m saving them for a future post, so stay on the lookout! Most of the books I read were personal biographies, a genre that I resonate with. There’s something comforting in listening to human stories, listening to how people learn, grow, and make the world around them better. To this end, I’m calling this summer the Summer of Stories. From making a scrollyteller of my Spotify listening patterns to listening to autobiographies to witnessing lives of people affected by the pandemic to telling my own narrative with this blog, this summer feels like it revolved around stories. I’ve learned that I love stories because they make me feel human. They make me feel like I can connect to the experiences of others. In summer-ry (get the joke?), I’d say this summer was pretty productive. I’ve picked up some cool new computer science tools and learned from valuable experiences. The optimist in me finds this reassuring that even in a pandemic, I can grow as a person.Building a Scrollytelling Visualization2020-10-11T00:00:00+00:002020-10-11T00:00:00+00:00https://vinaybhaip.com/blog/2020/10/11/spotify-vis-intro<p>In my last post, I built a <a href="/blog/2020/08/22/spotify-artist-viz">visualization of my Spotify data</a>. I used the data to create a scrollyteller, which presents a story as you scroll (if you check out the post, you’ll see what I mean). If you found this interesting and wanted to see how I made it, look no further! Please keep in mind this was my first scrollyteller, but hopefully you’ll find this to be insightful.</p>
<p>There’s two main parts to this project - the graphs and the formatting for the story.</p>
<h2 id="d3-visualizations">D3 Visualizations</h2>
<p>What’s D3? It stands for <em>Data-Driven Documents</em>. A lot of people think of it as a graphing library, but it’s more than that; it’s a manipulation of data at low-level. It might not seem like much at first, but after working with it for a little bit you start to understand what makes it special.</p>
<p>The best D3 tutorial I used was <a href="http://square.github.io/intro-to-d3/">this one from Square</a>. It walks you through the basics of using D3 and all in all was a great introduction.</p>
<p>Why D3 over other libraries? Why not just use Tableau or Excel? Once you look through <a href="https://observablehq.com/@d3/gallery">some projects that used D3</a>, you’ll want to learn it. Seriously, this library is amazing. There are other libraries that fulfill somewhat similar purposes, but D3 is by far the most popular.</p>
<h3 id="creating-a-bar-chart">Creating a Bar Chart</h3>
<p>Let’s go through how I built a simple bar chart for the Spotify Visualization post. In that post, I took my Spotify data and mapped it to a barchart for how long I listened to each artist. I’ll walkthrough the code, but I won’t give the data because I think it’s best if you find some other dataset and try to build it yourself (this just means you have to do a little bit more work instead of copy pasting code, but you’ll learn more!).</p>
<p>First, let’s create a bar chart. In something like Excel, this takes a couple of clicks, but in D3, we have to build it from scratch. First, we need to create a “canvas,” if you will, that we can draw the bar chart on. We do this with <a href="https://en.wikipedia.org/wiki/Scalable_Vector_Graphics">Scalable Vector Graphics (SVGs)</a>.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">var</span> <span class="nx">svg</span> <span class="o">=</span> <span class="nx">d3</span><span class="p">.</span><span class="nx">select</span><span class="p">(</span><span class="dl">'</span><span class="s1">.chart</span><span class="dl">'</span><span class="p">)</span>
<span class="p">.</span><span class="nx">append</span><span class="p">(</span><span class="dl">'</span><span class="s1">svg</span><span class="dl">'</span><span class="p">)</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">'</span><span class="s1">width</span><span class="dl">'</span><span class="p">,</span> <span class="nx">width</span><span class="p">)</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">'</span><span class="s1">height</span><span class="dl">'</span><span class="p">,</span> <span class="nx">height</span><span class="p">)</span>
</code></pre></div></div>
<p>You’ll notice here we’ve “chained” together several functions; we can do this because when we call each function, it returns an instance of the object itself. So here, we select the HTML element with the class ‘chart,’ add on an SVG element, and then adjust the width and height for our canvas.</p>
<p>Now that we have our canvas, the logical next step is axes. To do this, let’s create some scales to build off. (Note, I say “scales” not “axes” here!)</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">var</span> <span class="nx">scaleArtistY</span> <span class="o">=</span> <span class="nx">d3</span><span class="p">.</span><span class="nx">scaleBand</span><span class="p">()</span>
<span class="p">.</span><span class="nx">domain</span><span class="p">(</span><span class="nx">data</span><span class="p">.</span><span class="nx">map</span><span class="p">((</span><span class="nx">d</span><span class="p">)</span><span class="o">=></span><span class="nx">d</span><span class="p">.</span><span class="nx">artist</span><span class="p">))</span>
<span class="p">.</span><span class="nx">range</span><span class="p">([</span><span class="nx">margin</span><span class="p">.</span><span class="nx">top</span><span class="p">,</span> <span class="nx">height</span><span class="o">-</span><span class="nx">margin</span><span class="p">.</span><span class="nx">bottom</span><span class="p">])</span>
<span class="kd">var</span> <span class="nx">scaleAmountX</span> <span class="o">=</span> <span class="nx">d3</span><span class="p">.</span><span class="nx">scaleLinear</span><span class="p">()</span>
<span class="p">.</span><span class="nx">domain</span><span class="p">([</span><span class="mi">0</span><span class="p">,</span> <span class="nx">d3</span><span class="p">.</span><span class="nx">max</span><span class="p">(</span><span class="nx">data</span><span class="p">,</span> <span class="p">(</span><span class="nx">d</span><span class="p">,</span><span class="nx">i</span><span class="p">)</span> <span class="o">=></span> <span class="nx">d</span><span class="p">.</span><span class="nx">total_time</span><span class="p">)])</span>
<span class="p">.</span><span class="nx">range</span><span class="p">([</span><span class="nx">margin</span><span class="p">.</span><span class="nx">left</span><span class="p">,</span> <span class="nx">width</span><span class="o">-</span><span class="nx">margin</span><span class="p">.</span><span class="nx">right</span><span class="p">])</span>
<span class="p">.</span><span class="nx">nice</span><span class="p">();</span>
<span class="c1">//nice rounds off values</span>
<span class="kd">var</span> <span class="nx">scaleAmountColor</span> <span class="o">=</span>
<span class="nx">d3</span><span class="p">.</span><span class="nx">scaleOrdinal</span><span class="p">(</span><span class="nx">d3</span><span class="p">.</span><span class="nx">quantize</span><span class="p">(</span><span class="nx">d3</span><span class="p">.</span><span class="nx">interpolateWarm</span><span class="p">,</span> <span class="mi">26</span><span class="p">))</span>
</code></pre></div></div>
<p>What’s a scale? Well, think of it as a function from the raw data dimension to the visualization data. Scales are functions that take in raw data, like what year it is for example, and return a concrete dimension for the display, like a pixel location.</p>
<p>We have three dimensions here. The first, <code class="language-plaintext highlighter-rouge">scaleArtistY</code>, maps a Spotify artist to a <em>y value</em>. This is what the domain and the range functions are doing, defining the input and outputs. <code class="language-plaintext highlighter-rouge">d3.scaleBand()</code> allows us to take categorical variables (like the names of Spotify artists we’re using) to “bands” (read more about it <a href="https://observablehq.com/@d3/d3-scaleband">here</a>).</p>
<p><code class="language-plaintext highlighter-rouge">scaleAmountX</code> is the quantifier for how long I’ve listened to each Spotify artist. It uses <code class="language-plaintext highlighter-rouge">scaleLinear()</code> because we’re mapping a continuous variable to another continuous range. <code class="language-plaintext highlighter-rouge">scaleAmountColor</code> is a way to make the graph look a little bit more aesthetically pleasing.</p>
<p>Next, we define our axes. Remember, this is different from our scales. Axes are the lines you see on your screen, scales are functions that map one value to another.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">var</span> <span class="nx">yAxis</span> <span class="o">=</span> <span class="nx">d3</span><span class="p">.</span><span class="nx">axisLeft</span><span class="p">(</span><span class="nx">scaleArtistY</span><span class="p">)</span>
<span class="kd">var</span> <span class="nx">xAxis</span> <span class="o">=</span> <span class="nx">d3</span><span class="p">.</span><span class="nx">axisBottom</span><span class="p">(</span><span class="nx">scaleAmountX</span><span class="p">)</span>
<span class="p">.</span><span class="nx">tickFormat</span><span class="p">(</span><span class="nx">val</span> <span class="o">=></span> <span class="nx">val</span> <span class="o">+</span> <span class="dl">"</span><span class="s2"> hrs</span><span class="dl">"</span><span class="p">)</span>
<span class="p">.</span><span class="nx">ticks</span><span class="p">(</span><span class="mi">4</span><span class="p">);</span>
</code></pre></div></div>
<p>Now that we’ve defined our axes, we need to draw them.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">//g stands for "group"</span>
<span class="nx">svg</span><span class="p">.</span><span class="nx">append</span><span class="p">(</span><span class="dl">'</span><span class="s1">g</span><span class="dl">'</span><span class="p">)</span>
<span class="p">.</span><span class="nx">style</span><span class="p">(</span><span class="dl">"</span><span class="s2">font-family</span><span class="dl">"</span><span class="p">,</span> <span class="nx">font</span><span class="p">)</span>
<span class="p">.</span><span class="nx">style</span><span class="p">(</span><span class="dl">"</span><span class="s2">font-size</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">1vw</span><span class="dl">"</span><span class="p">)</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">'</span><span class="s1">class</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">y axis</span><span class="dl">'</span><span class="p">)</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">"</span><span class="s2">transform</span><span class="dl">"</span><span class="p">,</span> <span class="s2">`translate(</span><span class="p">${</span><span class="nx">margin</span><span class="p">.</span><span class="nx">left</span><span class="p">}</span><span class="s2">,0)`</span><span class="p">)</span>
<span class="p">.</span><span class="nx">call</span><span class="p">(</span><span class="nx">yAxis</span><span class="p">);</span>
<span class="nx">svg</span><span class="p">.</span><span class="nx">append</span><span class="p">(</span><span class="dl">"</span><span class="s2">g</span><span class="dl">"</span><span class="p">)</span>
<span class="p">.</span><span class="nx">style</span><span class="p">(</span><span class="dl">"</span><span class="s2">font-family</span><span class="dl">"</span><span class="p">,</span> <span class="nx">font</span><span class="p">)</span>
<span class="p">.</span><span class="nx">style</span><span class="p">(</span><span class="dl">"</span><span class="s2">font-size</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">1vw</span><span class="dl">"</span><span class="p">)</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">'</span><span class="s1">class</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">x axis</span><span class="dl">'</span><span class="p">)</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">"</span><span class="s2">transform</span><span class="dl">"</span><span class="p">,</span> <span class="s2">`translate(0,</span><span class="p">${</span><span class="nx">height</span> <span class="o">-</span> <span class="nx">margin</span><span class="p">.</span><span class="nx">bottom</span><span class="p">}</span><span class="s2">)`</span><span class="p">)</span>
<span class="p">.</span><span class="nx">call</span><span class="p">(</span><span class="nx">xAxis</span><span class="p">);</span>
</code></pre></div></div>
<p>Great! The next step, you might think is to draw the content of the graphs. But before we get there, let’s define some helper variables. There’s an important concept called data binding in D3, but this walkthrough’s goal in brevity will omit this. Data binding is a very important process in D3, so I highly recommend doing some research on this on your own (the Square tutorial above helps out with this).</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">var</span> <span class="nx">barg</span> <span class="o">=</span> <span class="nx">svg</span><span class="p">.</span><span class="nx">append</span><span class="p">(</span><span class="dl">"</span><span class="s2">g</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">rects</span> <span class="o">=</span> <span class="nx">barg</span>
<span class="p">.</span><span class="nx">selectAll</span><span class="p">(</span><span class="dl">"</span><span class="s2">rect</span><span class="dl">"</span><span class="p">)</span>
<span class="p">.</span><span class="nx">data</span><span class="p">(</span><span class="nx">data</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">newRects</span> <span class="o">=</span> <span class="nx">rects</span><span class="p">.</span><span class="nx">enter</span><span class="p">();</span>
<span class="kd">var</span> <span class="nx">texts</span> <span class="o">=</span> <span class="nx">barg</span>
<span class="p">.</span><span class="nx">selectAll</span><span class="p">(</span><span class="dl">"</span><span class="s2">text</span><span class="dl">"</span><span class="p">)</span>
<span class="p">.</span><span class="nx">data</span><span class="p">(</span><span class="nx">data</span><span class="p">)</span>
<span class="kd">var</span> <span class="nx">newTexts</span> <span class="o">=</span> <span class="nx">texts</span><span class="p">.</span><span class="nx">enter</span><span class="p">();</span>
</code></pre></div></div>
<p>Now we’ve got everything setup, it’s time to draw the actual bars!</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">newRects</span>
<span class="p">.</span><span class="nx">append</span><span class="p">(</span><span class="dl">"</span><span class="s2">rect</span><span class="dl">"</span><span class="p">)</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">"</span><span class="s2">class</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">rect</span><span class="dl">"</span><span class="p">)</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">"</span><span class="s2">id</span><span class="dl">"</span><span class="p">,</span> <span class="p">(</span><span class="nx">d</span><span class="p">,</span><span class="nx">i</span><span class="p">)</span> <span class="o">=></span> <span class="nx">i</span><span class="p">)</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">"</span><span class="s2">x</span><span class="dl">"</span><span class="p">,</span> <span class="nx">margin</span><span class="p">.</span><span class="nx">left</span> <span class="o">+</span> <span class="mi">2</span><span class="p">)</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">"</span><span class="s2">y</span><span class="dl">"</span><span class="p">,</span> <span class="p">(</span><span class="nx">d</span><span class="p">,</span><span class="nx">i</span><span class="p">)</span> <span class="o">=></span> <span class="nx">scaleArtistY</span><span class="p">(</span><span class="nx">d</span><span class="p">.</span><span class="nx">artist</span><span class="p">))</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">"</span><span class="s2">width</span><span class="dl">"</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">"</span><span class="s2">height</span><span class="dl">"</span><span class="p">,</span> <span class="p">(</span><span class="nx">d</span><span class="p">,</span><span class="nx">i</span><span class="p">)</span> <span class="o">=></span> <span class="nx">scaleArtistY</span><span class="p">.</span><span class="nx">bandwidth</span><span class="p">())</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">"</span><span class="s2">fill</span><span class="dl">"</span><span class="p">,</span> <span class="p">(</span><span class="nx">d</span><span class="p">,</span><span class="nx">i</span><span class="p">)</span> <span class="o">=></span> <span class="nx">scaleAmountColor</span><span class="p">(</span><span class="nx">d</span><span class="p">.</span><span class="nx">total_time</span><span class="p">))</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">"</span><span class="s2">opacity</span><span class="dl">"</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
</code></pre></div></div>
<p><img src="/blog/assets/images/d3-bar-graph-basics/empty-graph.png" alt="Empty Graph" /></p>
<p>Hmm, this is just empty. Looking at the previous code, we set the width to 0. Why? Well let’s add in the rest of the code and see.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">barg</span><span class="p">.</span><span class="nx">selectAll</span><span class="p">(</span><span class="dl">"</span><span class="s2">rect</span><span class="dl">"</span><span class="p">)</span>
<span class="p">.</span><span class="nx">transition</span><span class="p">()</span>
<span class="p">.</span><span class="nx">duration</span><span class="p">(</span><span class="mi">1000</span><span class="p">)</span>
<span class="p">.</span><span class="nx">attr</span><span class="p">(</span><span class="dl">"</span><span class="s2">width</span><span class="dl">"</span><span class="p">,</span> <span class="p">(</span><span class="nx">d</span><span class="p">,</span><span class="nx">i</span><span class="p">)</span> <span class="o">=></span> <span class="nx">scaleAmountX</span><span class="p">(</span><span class="nx">d</span><span class="p">.</span><span class="nx">total_time</span><span class="p">)</span> <span class="o">-</span> <span class="nx">margin</span><span class="p">.</span><span class="nx">left</span><span class="p">)</span>
</code></pre></div></div>
<p>Here, we’ve added an <em>transition</em> to the rectangle width. We now have an animation for the bar graph; let’s check it out!</p>
<p><img src="/blog/assets/images/d3-bar-graph-basics/animated-bar.gif" alt="Bar Graph" />
<em>Notice I’ve added text elements to the graph, but it’s nothing too different from what I had done before.</em></p>
<p>Any plain old graph making software like Excel would not have given us all this flexibility. Yes, there’s some more organization to do beforehand, but it gives us the tools to do whatever we desire. This is because at the core, D3 is not a graphing library but a data manipulation library.</p>
<p>But D3 goes beyond just some fancy animations. There’s so much that can be done with it, and this is only the beginning.</p>
<h2 id="scrollytelling-and-more">Scrollytelling and more</h2>
<p>If you notice on the Spotify Visualization post, the charts animate as you scroll down the page. On the left side, there’s some updating text that describes the visualization as it changes. This beautiful way of demonstrating charts is known as scrollytelling, an aptly chosen portmanteau.</p>
<p>The library I used for this was <a href="https://github.com/russellgoldenberg/scrollama">Scrollama</a>, and there is some example code in the repository. A good guide on making these scrollytellers was <a href="https://pudding.cool/process/responsive-scrollytelling/">this article from the Pudding</a>.</p>
<p>This post does not nearly explain everything that I did when making my Spotify Visualization scrollyteller. If you want to see the full code for this, check it out <a href="https://github.com/vbhaip/spotify-artist-vis">here</a>. Feel free to reach out if you’ve got any questions!</p>
<p>Hopefully, this post has given you a brief look into what making a scrollyteller entails by showing a simple example alongside with the resources that I’ve used. My personal D3 explorations are just getting started, so stay on the look out for more interesting posts!</p>
<!--
```javascript
newTexts
.append("text")
.attr("text-anchor", "end")
.attr("y", (d,i) => scaleArtistY(d.artist) + scaleArtistY.bandwidth()/2.0 + 3)
.attr("x", (d,i) => scaleAmountX(d.total_time))
.text((d,i) => d3.format(".1f")(d.total_time))
.attr("fill", "white")
.style("font-family", font)
.style("font-size", "1vw")
```
-->In my last post, I built a visualization of my Spotify data. I used the data to create a scrollyteller, which presents a story as you scroll (if you check out the post, you’ll see what I mean). If you found this interesting and wanted to see how I made it, look no further! Please keep in mind this was my first scrollyteller, but hopefully you’ll find this to be insightful. There’s two main parts to this project - the graphs and the formatting for the story. D3 Visualizations What’s D3? It stands for Data-Driven Documents. A lot of people think of it as a graphing library, but it’s more than that; it’s a manipulation of data at low-level. It might not seem like much at first, but after working with it for a little bit you start to understand what makes it special. The best D3 tutorial I used was this one from Square. It walks you through the basics of using D3 and all in all was a great introduction. Why D3 over other libraries? Why not just use Tableau or Excel? Once you look through some projects that used D3, you’ll want to learn it. Seriously, this library is amazing. There are other libraries that fulfill somewhat similar purposes, but D3 is by far the most popular. Creating a Bar Chart Let’s go through how I built a simple bar chart for the Spotify Visualization post. In that post, I took my Spotify data and mapped it to a barchart for how long I listened to each artist. I’ll walkthrough the code, but I won’t give the data because I think it’s best if you find some other dataset and try to build it yourself (this just means you have to do a little bit more work instead of copy pasting code, but you’ll learn more!). First, let’s create a bar chart. In something like Excel, this takes a couple of clicks, but in D3, we have to build it from scratch. First, we need to create a “canvas,” if you will, that we can draw the bar chart on. We do this with Scalable Vector Graphics (SVGs). var svg = d3.select('.chart') .append('svg') .attr('width', width) .attr('height', height) You’ll notice here we’ve “chained” together several functions; we can do this because when we call each function, it returns an instance of the object itself. So here, we select the HTML element with the class ‘chart,’ add on an SVG element, and then adjust the width and height for our canvas. Now that we have our canvas, the logical next step is axes. To do this, let’s create some scales to build off. (Note, I say “scales” not “axes” here!) var scaleArtistY = d3.scaleBand() .domain(data.map((d)=>d.artist)) .range([margin.top, height-margin.bottom]) var scaleAmountX = d3.scaleLinear() .domain([0, d3.max(data, (d,i) => d.total_time)]) .range([margin.left, width-margin.right]) .nice(); //nice rounds off values var scaleAmountColor = d3.scaleOrdinal(d3.quantize(d3.interpolateWarm, 26)) What’s a scale? Well, think of it as a function from the raw data dimension to the visualization data. Scales are functions that take in raw data, like what year it is for example, and return a concrete dimension for the display, like a pixel location. We have three dimensions here. The first, scaleArtistY, maps a Spotify artist to a y value. This is what the domain and the range functions are doing, defining the input and outputs. d3.scaleBand() allows us to take categorical variables (like the names of Spotify artists we’re using) to “bands” (read more about it here). scaleAmountX is the quantifier for how long I’ve listened to each Spotify artist. It uses scaleLinear() because we’re mapping a continuous variable to another continuous range. scaleAmountColor is a way to make the graph look a little bit more aesthetically pleasing. Next, we define our axes. Remember, this is different from our scales. Axes are the lines you see on your screen, scales are functions that map one value to another. var yAxis = d3.axisLeft(scaleArtistY) var xAxis = d3.axisBottom(scaleAmountX) .tickFormat(val => val + " hrs") .ticks(4); Now that we’ve defined our axes, we need to draw them. //g stands for "group" svg.append('g') .style("font-family", font) .style("font-size", "1vw") .attr('class', 'y axis') .attr("transform", `translate(${margin.left},0)`) .call(yAxis); svg.append("g") .style("font-family", font) .style("font-size", "1vw") .attr('class', 'x axis') .attr("transform", `translate(0,${height - margin.bottom})`) .call(xAxis); Great! The next step, you might think is to draw the content of the graphs. But before we get there, let’s define some helper variables. There’s an important concept called data binding in D3, but this walkthrough’s goal in brevity will omit this. Data binding is a very important process in D3, so I highly recommend doing some research on this on your own (the Square tutorial above helps out with this). var barg = svg.append("g"); var rects = barg .selectAll("rect") .data(data); var newRects = rects.enter(); var texts = barg .selectAll("text") .data(data) var newTexts = texts.enter(); Now we’ve got everything setup, it’s time to draw the actual bars! newRects .append("rect") .attr("class", "rect") .attr("id", (d,i) => i) .attr("x", margin.left + 2) .attr("y", (d,i) => scaleArtistY(d.artist)) .attr("width", 0) .attr("height", (d,i) => scaleArtistY.bandwidth()) .attr("fill", (d,i) => scaleAmountColor(d.total_time)) .attr("opacity", 1) Hmm, this is just empty. Looking at the previous code, we set the width to 0. Why? Well let’s add in the rest of the code and see. barg.selectAll("rect") .transition() .duration(1000) .attr("width", (d,i) => scaleAmountX(d.total_time) - margin.left) Here, we’ve added an transition to the rectangle width. We now have an animation for the bar graph; let’s check it out! Notice I’ve added text elements to the graph, but it’s nothing too different from what I had done before. Any plain old graph making software like Excel would not have given us all this flexibility. Yes, there’s some more organization to do beforehand, but it gives us the tools to do whatever we desire. This is because at the core, D3 is not a graphing library but a data manipulation library. But D3 goes beyond just some fancy animations. There’s so much that can be done with it, and this is only the beginning. Scrollytelling and more If you notice on the Spotify Visualization post, the charts animate as you scroll down the page. On the left side, there’s some updating text that describes the visualization as it changes. This beautiful way of demonstrating charts is known as scrollytelling, an aptly chosen portmanteau. The library I used for this was Scrollama, and there is some example code in the repository. A good guide on making these scrollytellers was this article from the Pudding. This post does not nearly explain everything that I did when making my Spotify Visualization scrollyteller. If you want to see the full code for this, check it out here. Feel free to reach out if you’ve got any questions! Hopefully, this post has given you a brief look into what making a scrollyteller entails by showing a simple example alongside with the resources that I’ve used. My personal D3 explorations are just getting started, so stay on the look out for more interesting posts!Visualized: My Spotify Listening Patterns2020-08-22T00:00:00+00:002020-08-22T00:00:00+00:00https://vinaybhaip.com/blog/2020/08/22/spotify-artist-viz<meta charset="utf-8" />
<!-- Load d3.js -->
<script src="https://d3js.org/d3.v5.js"></script>
<script src="https://d3js.org/d3-array.v2.min.js"></script>
<script src="https://d3js.org/topojson.v1.min.js"></script>
<script src="https://rawgit.com/susielu/d3-annotation/master/d3-annotation.min.js"></script>
<script src="https://unpkg.com/intersection-observer"></script>
<script src="https://unpkg.com/scrollama"></script>
<link rel="stylesheet" href="/blog/assets/spotify-artist-analysis/style.css" />
<link rel="manifest" href="/blog/assets/spotify-artist-analysis/manifest.json" />
<h1 id="rotate">Please put your device in landscape mode.</h1>
<div id="scroll">
<h4>Which artists do I listen to the most? How has my listening patterns changed over time? Scroll down to reveal the story!</h4>
<div class="scroll__graphic">
<div class="chart">
</div>
</div>
<div class="scroll__text">
<div class="step" data-step="-1">
<p>I recently uncovered my Spotify data documenting every song I've listened to for the past two years. Let's dive in.</p>
</div>
<div class="step" data-step="1">
<p>This figure displays the cumulative time I spent listening to each artist's music. The artists I listen to fall into three tiers.</p>
</div>
<div class="step" data-step="1.1">
<p>At the top of the list (with nearly 50 hours each!) are Drake and J. Cole. Personally, my favorite artist is J. Cole and I listen to a lot of the mainstream Drake songs, so this makes sense. No surprises here. </p>
</div>
<div class="step" data-step="1.2">
<p>Kanye, Maroon 5, and Post Malone round up tier 2 at about 25 hours a piece. I'm surprised with Post Malone's appearance here—I didn't realize I listened to him <i>that</i> much.</p>
</div>
<div class="step" data-step="1.3">
<p>And the rest trail behind below the 20 hour marker (Juice WRLD and Kendrick are close though!). It's worth pointing out that one of these artists is actually a podcast—Hello Internet.</p>
</div>
<div class="step" data-step="1.4">
<p>This information is interesting by itself, but this neglects my listening patterns over time. Let's look more into the time dimension.</p>
</div>
<div class="step" data-step="2">
<p>This shows a cumulative plot of which artists I listened to over time. Hover over each line to see which artist it corresponds to!</p>
</div>
<div class="step" data-step="2.1">
<p>Drake and J. Cole have a fairly consistent growth pattern.</p>
</div>
<div class="step" data-step="2.2">
<p>Post Malone has a huge growth in late 2019. Why? <i>Hollywood's Bleeding</i> dropped at the start of September—makes perfect sense.</p>
</div>
<div class="step" data-step="2.3">
<p>Juice WRLD has a prolific rise after he passes away. Really wish I had listened to his music earlier; turns out his music is 🔥.</p>
</div>
<div class="step" data-step="2.4">
<p>Maroon 5 has an insane increase to 25 hours in the matter of a few months. This happened toward the middle of quarantine when I was vibing with Maroon 5 on repeat. The almost vertical line captures just how much I listened to Adam Levine.</p>
</div>
<div class="step" data-step="3">
<p>And that's it! There's still more interesting information in the data I didn't look at. For example, I chose only to look at the top 25 artists that I've listened to, leaving rich information behind, but that's for another day. You can download your own Spotify history <a href="https://www.spotify.com/uk/account/privacy/">here</a>.</p>
</div>
</div>
<script src="/blog/assets/spotify-artist-analysis/index.js"></script>
</div>Please put your device in landscape mode. Which artists do I listen to the most? How has my listening patterns changed over time? Scroll down to reveal the story! I recently uncovered my Spotify data documenting every song I've listened to for the past two years. Let's dive in. This figure displays the cumulative time I spent listening to each artist's music. The artists I listen to fall into three tiers. At the top of the list (with nearly 50 hours each!) are Drake and J. Cole. Personally, my favorite artist is J. Cole and I listen to a lot of the mainstream Drake songs, so this makes sense. No surprises here. Kanye, Maroon 5, and Post Malone round up tier 2 at about 25 hours a piece. I'm surprised with Post Malone's appearance here—I didn't realize I listened to him that much. And the rest trail behind below the 20 hour marker (Juice WRLD and Kendrick are close though!). It's worth pointing out that one of these artists is actually a podcast—Hello Internet. This information is interesting by itself, but this neglects my listening patterns over time. Let's look more into the time dimension. This shows a cumulative plot of which artists I listened to over time. Hover over each line to see which artist it corresponds to! Drake and J. Cole have a fairly consistent growth pattern. Post Malone has a huge growth in late 2019. Why? Hollywood's Bleeding dropped at the start of September—makes perfect sense. Juice WRLD has a prolific rise after he passes away. Really wish I had listened to his music earlier; turns out his music is 🔥. Maroon 5 has an insane increase to 25 hours in the matter of a few months. This happened toward the middle of quarantine when I was vibing with Maroon 5 on repeat. The almost vertical line captures just how much I listened to Adam Levine. And that's it! There's still more interesting information in the data I didn't look at. For example, I chose only to look at the top 25 artists that I've listened to, leaving rich information behind, but that's for another day. You can download your own Spotify history here.Overhauling My Website2020-07-18T00:00:00+00:002020-07-18T00:00:00+00:00https://vinaybhaip.com/blog/2020/07/18/website-overhaul<!--more-->
<p>In sophomore year of high school, I built my personal website from scratch. Why? I think it’s nice to have your own corner on the internet. This was the first project where I created a plan and followed through. At the time, it looked amazing.</p>
<p><img src="/blog/assets/images/overhaul-website/old-website.png" alt="Old Website" />
<em>The first version of this website. Check it out <a href="https://vinaybhaip.com/old">here</a>.</em></p>
<p>The problem - <strong>time fractures our perspectives</strong>. To my current set of eyes, this website looks - to be quite frank - bland, uninformative, and disappointing.</p>
<h3 id="goals">Goals</h3>
<p>If I wanted to rebuild this website, the first step would be identifying what made it so bad. So I hit the whiteboard:</p>
<ul>
<li>The theme colors were dull. Grey, navy, white, and red just don’t stick out. Like c’mon 2017 Vinay, show your personality a bit more.</li>
<li>There was no information about who I was. All the website said was that I liked coding and machine learning - nothing that’s <em>unique</em>.</li>
<li>The cool features of the website didn’t even work! The website you see above had animated gears rotating, the arrows floating, and a sick logo animation - none of it worked anymore. (I had used <a href="https://spiritapp.io/">Spirit</a> while it was in beta, but now it required a subscription I did not want to pay).</li>
<li>There was no way to contact me. I had intended to have a contact form when designing the website, but I kept putting it off.</li>
<li>Everything felt clunky and straight-up <em>wrong</em>. It wasn’t true to who I was. The font I used felt generic and the buttons were large and childish.</li>
</ul>
<p>In sum: 🤢.</p>
<p>Given everything I didn’t like, I wrote out what I wanted for version 2.0:</p>
<ul>
<li>Simple, elegant, and uniquely me</li>
<li>Blog (Why? See <a href="https://vinaybhaip.com/blog/2020/06/20/words">here</a>)</li>
<li>Responsive design (should look nice on your phone)</li>
<li>Contact me page</li>
<li>Showcase my design and development skills</li>
<li>Custom domain</li>
</ul>
<p>The objectives for this were a bit broad, but at least I had more concrete goals.</p>
<h3 id="prototyping">Prototyping</h3>
<p>As with all websites I make, I designed some mockups using <a href="https://affinity.serif.com/en-us/designer/">Affinity Designer</a>.</p>
<p><img src="/blog/assets/images/overhaul-website/website-v2.png" alt="Website v2" />
<em>The first version of the website that I mocked up. Keep in mind this is only the front page, and there would be more below.</em></p>
<p>I created this mockup about a year before I came up with the objectives for my website, hence the lackluster. To be honest, this might be worse than the first version of the website. Words that come to mind: dull, boring, clichéd. “Developer, Designer, Dreamer” sounds so unoriginal and conveys nothing.</p>
<p><img src="/blog/assets/images/overhaul-website/website-v3.png" alt="Website v3" />
<em>The second version of the website that I mocked up.</em></p>
<p>Again, I don’t really see any improvement. The font choice makes me wince. I’m glad I never actually made these mockups. While it’s marginally better than the previous version, it somehow conveys even less information.</p>
<p>These designs were <strong>bad</strong>, no sugar-coating it. If attach my name to anything, it should be high-quality. At this point I stepped aside and came back a year later once I came up with the concrete objectives above.</p>
<p>I also did more research to gain some inspiration online: personal favorites include <a href="https://caferati.me/">Rafael Caferati’s website</a> and <a href="http://danielautry.com/">Daniel Autry’s website</a>.</p>
<p><img src="/blog/assets/images/overhaul-website/website-v4.png" alt="Website v4" />
<em>The third version of the website that I mocked up. It’s actually looking good now!</em></p>
<p>I don’t know about you but it is not an understatement to say this is <strong>miles</strong> better. And look! It actually shows my face and has a pleasing color palette. It’s amazing to see how much I’ve improved in a year.</p>
<p>Breaking down some of the design choices: First, it uses a diagonal line to create a juxtaposition between two colors. I like this because visually it pops out more and creates a clear division between the menu and a description of me. I also love how I <em>show</em> who I am, more than just a “developer.” The logo in the top-left corner is also completely different from before, but how I got there is a story for another day. The color scheme is a tribute to the next chapter of my life (go Hoos!).</p>
<p>And this was only the front page. After some tweaking here and there, I was ready to materialize my vision.</p>
<h3 id="development">Development</h3>
<p>I built the website as a <a href="https://en.wikipedia.org/wiki/Static_web_page">static web page</a> so I could host it on <a href="https://pages.github.com/">Github Pages</a>. Slowly but steadily, everything came together: first the backbone, then some nice animations to liven the website. I tried avoiding external dependencies that might not work years down the line to keep the website “future-proof.”</p>
<p>After about a week or so of tinkering, I was done! You can check out the whole website at <a href="https://vinaybhaip.com">https://vinaybhaip.com</a>. I’m pretty proud of how it all turned out (and it’s pretty close to the mockup!).</p>
<p>After creating this website, I bought my domain through Google Domains and linked it up to <a href="https://vbhaip.github.io">https://vbhaip.github.io</a>. I chose to host on Github instead of another host like GoDaddy to minimize my costs (and because all the services other hosts marketed seemed to drive the price up - with Google Domains it was pretty clear cut).</p>
<p>If I could go back in time and change one thing about the design, it reluctantly would be the diagonals I used. Turns out formatting items with steep diagonals is hard, especially for smaller devices like phones. While I think they look amazing, I think the cost-benefit tradeoff for the two diagonals I used in the website is questionable.</p>
<h3 id="final-thoughts">Final Thoughts</h3>
<p>When I made this website a few years ago, I was extremely proud, as I should’ve been - I worked super hard. After some time this pride shifted to cringe - was <em>this</em> the best work I did? I think it’s good to cringe at your former self - it’s a sign that I’ve grown in my aesthetic, development skills, and more broadly my beliefs.</p>
<p>If I don’t cringe at my past self and want to improve, <strong>that’s</strong> a problem.</p>
<p>So in a couple of years, don’t be surprised if this website changes. It’s just me growing.</p>How I Built HelioHex2020-07-05T00:00:00+00:002020-07-05T00:00:00+00:00https://vinaybhaip.com/blog/2020/07/05/heliohex<!--more-->
<p>For the past three months, I’ve been building a highly configurable, modular, hexagon-based lighting system from scratch that is controllable from any device and syncs to Spotify. Here’s a demo:</p>
<div class="video-container">
<iframe width="560" height="315" src="https://www.youtube.com/embed/8TQva79vo88" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
</div>
<p>Here’s some of the highlights of HelioHex:</p>
<ul>
<li>Different lighting ‘modes’
<ul>
<li>Set the color of the whole display</li>
<li>Set the color of individual hexagons</li>
<li>Generate a random aesthetically-pleasing color palette</li>
<li>Sync the color of the display to the time of the day</li>
</ul>
</li>
<li>Spotify integration
<ul>
<li>Matches up the colors of each hexagon to the features of the song playing</li>
<li>Changes depending on the emotions of the song e.g. sadder songs are more blue</li>
</ul>
</li>
<li>Adaptable web controller
<ul>
<li>Use any device on the network to control the device</li>
<li>Control all the different modes and brightness of the display</li>
<li>Shows a virtual visualization of the display</li>
</ul>
</li>
<li>Flexible design
<ul>
<li>Arrange the hexagons in whatever pattern you like - change the settings in one file and everything automatically updates</li>
<li>Expand the structure easily if you build more hexagons</li>
</ul>
</li>
</ul>
<h2 id="table-of-contents">Table of Contents</h2>
<ol>
<li><a href="#inspiration">Inspiration</a></li>
<li><a href="#research">Research</a></li>
<li><a href="#vision">Vision</a></li>
<li><a href="#construction">Construction</a>
<ol>
<li><a href="#materials">Materials</a></li>
<li><a href="#wooden-frames">Wooden Frames</a></li>
<li><a href="#light-diffusion">Light Diffusion</a></li>
</ol>
</li>
<li><a href="#electronics">Electronics</a>
<ol>
<li><a href="#wiring-lights">Wiring Lights</a></li>
<li><a href="#supplying-current">Supplying Current</a></li>
<li><a href="#extending-wires">Extending Wires</a></li>
</ol>
</li>
<li><a href="#software">Software</a>
<ol>
<li><a href="#spotify-syncing">Spotify Syncing</a></li>
<li><a href="#web-app-controller">Web App Controller</a>
<!-- 3. [Starting on Boot](#starting-on-boot) --></li>
</ol>
</li>
<li><a href="#final-touches">Final Touches</a></li>
<li><a href="#reflections">Reflections</a></li>
</ol>
<p><br /></p>
<h2 id="inspiration">Inspiration</h2>
<p>Late March, I realized that all my schoolwork and testing would soon be over. In an effort to avoid spending my quarantine solely indulging in video games and Netflix, I sought a project.</p>
<p>While brainstorming ideas, I came across the <a href="https://nanoleaf.me/en/consumer-led-lighting/products/smarter-series/nanoleaf-light-panels-smarter-kit/">Nanoleaf Light Panels</a>, a vibrant lighting system that gamers frequently display in their rooms. I thought that I could extend the functionality of this to show lights based off events, like if I received a notification on my phone. And maybe Facebook notifications could be blue, Snapchat notifications yellow, and email notifications red. All of the pieces of this project were starting to come together until I saw the pricetag: <strong>$200</strong>.</p>
<p>Nope. No way was I gonna spend $200 on some lights. To make it worse, that was the price for nine measely panels. Don’t get me wrong - it looked impressive, just not worth a little over $20 per panel. And even if I still wanted it, the $200 set was sold out - I’d have to cough up $300 for fifteen panels instead.</p>
<p>Perhaps the coolest application of the Nanoleaf Light Panels is the <a href="https://www.youtube.com/watch?v=o41emqmX6ds">Rhythm Edition</a>, which syncs your lighting system to music. Watching the demo left me… disappointed. It was highly customizable, but perhaps too customizable. I just wanted to click a button and have the lighting system sync up to all the nuances in the song - if the song was going to have the bass drop, I wanted to emphasize that and to make the lights to feel alive. Maybe I’m in the minority here, but I was pretty unsatisfied - especially considering I’d be shelling out hundreds of dollars for this.</p>
<p>Then something clicked; why don’t I just build my own system? I could customize it to my heart’s desire and I could make it for a lot cheaper than $200. I would also learn a lot from this project - it seemed perfect. And so, I set out to build my own lighting system - HelioHex.</p>
<h2 id="research">Research</h2>
<p>As should be with any project, the first step was research. <a href="https://www.youtube.com/watch?v=2OP-oEbzB6g">This DIY video from NerdForge</a> looked very similar to what I wanted. The key features I wanted to adopt from this was the modularity, the hexagon design (because it looked really cool), and the remote controller. However, I felt the app he developed didn’t make the most of the lights - specifically, I wanted to sync up my own system with Spotify. Also, he 3D-printed the frame for the hexagons. Unfortunately, I do not have the luxury of a 3D-printer (and I couldn’t 3D-print at the library due to the lockdown). This motivated me to alternatively make a wooden frame to give off a sleek, less-manufactured look.</p>
<p>I also <a href="https://www.youtube.com/watch?v=7vGa1oAYAX8">checked out</a> <a href="https://www.youtube.com/watch?v=PwmheNfCnXM">many</a> <a href="https://www.instructables.com/id/DIY-Hexagonal-Nanoleaf-LED-Light/">more</a> <a href="https://www.thingiverse.com/thing:3354082">guides</a> that helped me frame my goals. Ultimately, this research helped me answer one big question: was this project attainable?</p>
<p>The answer was a confident ¯\<em>(ツ)</em>/¯. A lot of the electronics scared me and the coding seemed decently formidable. I like challenges, though, so I went for it. Also, given that I had boatloads of free time, it seemed like I had nothing better to do.</p>
<h2 id="vision">Vision</h2>
<p>The next step for me was to create a concrete vision of my desired end-result. First up were the non-negotiables - things my project HAD to have to consider it successful:</p>
<ul>
<li>Working lights that are easily controllable by a remote controller</li>
<li>Seamless integration with a music app</li>
<li>Modularity - changing the arrangement of the hexagon units should be relatively easy</li>
<li>Self containment - each unit is independent of other units</li>
<li>Preset lighting modes that “look nice”</li>
<li>A clean UI for the remote controller</li>
</ul>
<p>Next up were some nice to haves - not needed, but would make me feel really accomplished</p>
<ul>
<li>No <a href="https://en.wikipedia.org/wiki/Hard_coding">hard coding</a></li>
<li>Anyone should be able to control the lighting system</li>
<li>Each individual hexagon “unit” can have their color set</li>
</ul>
<p>Some of these goals were scary, especially when you combine them together - having the ability to set individual hexagon colors without hardcoding while having the UI of the remote controller look clean? Yeesh.</p>
<p>The next step was materializing this vision. What helped me was using a <a href="https://en.wikipedia.org/wiki/Kanban_board">Kanban Board</a>. I wrote mini tasks for myself to reach different goals - divided into sections like software, hardware, and construction and prioritized by importance. A Kanban Board made this ambitious project much more digestable. It also made sure that I wouldn’t spend time on unnecessary, small fixes unless they were needed. The goal was to build the system as quickly as possible, and then go back and optimize it. Basically the <a href="https://en.wikipedia.org/wiki/Agile_software_development">Agile Development Technique</a>. Totally see how helpful it is now and why companies emphasize it so heavily.</p>
<h2 id="construction">Construction</h2>
<h3 id="materials">Materials</h3>
<table>
<thead>
<tr>
<th>Material</th>
<th>Cost</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://www.amazon.com/CHINLY-Individually-Addressable-Waterproof-waterproof/dp/B06XNJSKXN/ref=sr_1_5">LED Strip</a></td>
<td>$29.90</td>
</tr>
<tr>
<td><a href="https://www.amazon.com/ALITOVE-Female-Connector-WS2812B-SK6812-RGBW/dp/B071H5XCN5/ref=sr_1_5">JST Connectors</a></td>
<td>$10.99</td>
</tr>
<tr>
<td><a href="https://www.amazon.com/dp/B075SXMD9Z/ref=sspa_dk_detail_3">RF Remote Controller</a></td>
<td>$8.99</td>
</tr>
<tr>
<td><a href="https://www.homedepot.com/p/Sande-Plywood-Common-5-2-mm-x-2-ft-x-4-ft-Actual-0-205-in-x-23-75-in-x-47-75-in-103064/202093790">Plywood</a></td>
<td>$10.96</td>
</tr>
<tr>
<td><a href="https://www.homedepot.com/p/OPTIX-23-75-in-x-47-75-in-White-Acrylic-Light-Panel-1A20084A/100564898">Acrylic Light Panel</a></td>
<td>$12.48</td>
</tr>
<tr>
<td><a href="https://www.microcenter.com/product/622539/raspberry-pi-4-model-b---8gb-ddr4">Raspberry Pi 4</a></td>
<td>$35.00</td>
</tr>
<tr>
<td><a href="https://www.amazon.com/CHINLY-Universal-Regulated-Switching-Transformer/dp/B01LZRIWZD/ref=sr_1_4">Power Supply</a></td>
<td>$20.99</td>
</tr>
<tr>
<td><a href="https://www.lowes.com/pd/Southwire-25-ft-14-AWG-Stranded-Black-Copper-THHN-Wire-By-the-Roll/1000992054">Wires</a> x3</td>
<td>$18.81</td>
</tr>
<tr>
<td><a href="https://www.homedepot.com/p/Legrand-5-ft-Non-Metallic-Raceway-Wire-Channel-White-NMW1/100038290">Cord Cover</a></td>
<td>$8.12</td>
</tr>
<tr>
<td>Hot glue, super glue, push pins, clamps, etc.</td>
<td>$30.00</td>
</tr>
<tr>
<td><strong>Total</strong></td>
<td><strong>$186.24</strong></td>
</tr>
</tbody>
</table>
<p>I got under my goal of $200! Ended up saving about $15 for three months of my time.</p>
<p>To be fair, I did buy other materials that I never ended up using and did not account for tax. This means that the <em>actual</em> cost of this project probably was $50-$100 higher, but we’ll just say it’s under $200 because it makes me feel better.</p>
<h3 id="wooden-frames">Wooden Frames</h3>
<p>The first legitimate step in the project was to create the casing for each unit. All the guides I found used 3D printed cases, which was inaccessible for me, so I used wood. I bought a 24” by 12” wooden sheet and cut it out into rectangular pieces. I used a hexagon shape for each unit so I had to glue six of these rectangles together. I hotglued a metal hinge into the shape of a 120° angle and used that as a reference to put the rectangles together. The structural integrity of each hexagon was important so I used a trifecta of super glue, hot glue, and wood glue.</p>
<p><img src="/blog/assets/images/heliohex/wooden-hexagon-construction.jpg" alt="Wooden Hexagon Construction" />
<em>Wooden hexagon construction. In the foreground, wooden side pairs are joined together that will eventually form full hexagons. In the background, you see that my worktable was a messy ping-pong table.</em></p>
<p>In the above picture, you’ll notice small triangles cut out at the bottom of each plywood piece. This allows for wires to pass through connecting units. In line with my goal of modularity, this cutout was on every side so I could configure my system however I’d like. I made eight of these units, the most I could make given the length of the LED strip.</p>
<h3 id="light-diffusion">Light Diffusion</h3>
<p>The goal of this lighting system was to have each hexagon display a uniform, individual color. To achieve this uniformity, the material on top of each hexagon must evenly diffuse light from the LED strip that lines the inner perimeter of each hexagon.</p>
<p>This is where the acrylic sheet comes in. The sheet I bought is basically the same material as the light panels that covers school ceilings.</p>
<p>The problem is that acrylic is very brittle to the point that any cut I made - using scissors, a saw, even with the knife Home Depot recommended - completely fragmented the acrylic. After spending a week trying to figure it out, I decided to hold it off till I could laser cut it when the lockdown lifted.</p>
<p>In the meanwhile, I used a poor man’s acrylic - paper. It served its purpose for how easy it was to use.</p>
<p><img src="/blog/assets/images/heliohex/paper-light.gif" alt="Paper Light" />
<em>Paper hexagons put on top of each unit. The lighting is controlled by a cheap RF remote.</em></p>
<p>Although it does look pretty cool, there’s a couple of problems. First, it’s not clean. Paper doesn’t give off that professional vibe I’m striving for and if I’m spending months on this, might as well make it look good. Second, paper is really flimsy. It wouldn’t stick well with tape, and I didn’t want a permanent solution like glue because I knew I wanted to change it eventually. Lastly, it doesn’t diffuse the light THAT well; it does the job, but it’s nothing amazing.</p>
<p><img src="/blog/assets/images/heliohex/acrylic-sheet-light.jpg" alt="Acrylic Sheet Light" />
<em>The uncut acrylic sheet on top of a lit unit.</em></p>
<p>These images don’t do justice to the acrylic sheet. It looks miles better with material that’s meant to diffuse light. Instead of waiting for a laser cutter for who knows how long, I got lucky when a friend allowed me to borrow their <a href="https://en.wikipedia.org/wiki/Circular_saw">hand-held circular saw</a>. The fine teeth on this allowed me to cut the acrylic with no worry.</p>
<p><img src="/blog/assets/images/heliohex/acrylic-sheet-after.jpg" alt="Acrylic Sheet After" />
<em>The acrylic sheet after I cut it. On the left of the sheet you can see the jaggedness resulting from my previous methods.</em></p>
<p>It turned out a success! I hotglued the cut-out acrylic pieces to the hexagons and shaved down the sharp edges.</p>
<h2 id="electronics">Electronics</h2>
<p>This is what I was worried about the most. Despite having taken multiple classes in high school dealing with electronics, I had no idea what I was doing - evident by me <em>almost</em> setting the house on fire a few times.</p>
<h3 id="wiring-lights">Wiring Lights</h3>
<p>I cut up the light strip to put 36 LEDs in each hexagon unit. Each LED draws 50mA at full brightness, meaning for eight hexagons, I would need 14.4 Amps as a current. This is <a href="https://www.asc.ohio-state.edu/physics/p616/safety/fatal_current.html">a lot</a> of current, so needless to say, I was a bit worried. I followed a <a href="https://tutorials-raspberrypi.com/connect-control-raspberry-pi-ws2812-rgb-led-strips/">simple tutorial</a> to wire the lights and it worked pretty well to my delight.</p>
<p>I then soldered <a href="https://en.wikipedia.org/wiki/JST_connector">JST connectors</a> to both ends of the LED strip in each unit. This would achieve my goal of modularity.</p>
<p><img src="/blog/assets/images/heliohex/basic-unit.jpg" alt="Basic Unit" />
<em>A basic unit with JST connectors soldered onto the ends.</em></p>
<h3 id="supplying-current">Supplying Current</h3>
<p>In my idealistic world, after I wired each unit, I wouldn’t have to ever touch the electronics again. Unfortunately, there wasn’t enough power going through all the hexagons.</p>
<p><img src="/blog/assets/images/heliohex/power-issues-before.jpg" alt="Power Issues Before" />
<em>All of the hexagons are set to white at max brightness, but insufficient power results in later hexagons only showing the red channel.</em></p>
<p>I needed some way to pass the 5V and the ground line to each hexagon without passing through the LEDs, which would draw power. My solution was allowing the power to bypass most LEDs in parallel by connecting the 5V and ground at the ends of the LED strip in each unit.</p>
<p><img src="/blog/assets/images/heliohex/power-issues-solution-on.jpg" alt="Power Issues Solution On" />
<em>The solution to the power issues. The blue wire connects the grounds and the red wire connects the 5V.</em></p>
<p>To my surprise, it worked!</p>
<p><img src="/blog/assets/images/heliohex/power-issues-after.jpg" alt="Power Issues After" />
<em>All of the hexagons are set to white at max brightness again. Not too bad!</em></p>
<p>You can still see some of the color fading away, but it’s much better. The reason it isn’t perfect is because the way I wired it requires the power to go through two LEDs in each unit before going to the next unit.</p>
<h3 id="extending-wires">Extending Wires</h3>
<p>My nightmares of soldering still didn’t end. In order to hang the display on the wall, I would need to extend the wires to at least to five feet. The problem was that when I soldered the longer wires, the LEDs started flickering randomly. Weirdly, when I touched the input data wire, everything worked perfectly.</p>
<p>After hours (and hours and hours) of research (the issue was <a href="https://electronics.stackexchange.com/questions/79752/what-is-common-mode-noise">common-mode noise</a>?), and a brief scare that I shorted and burnt all of the LEDs 😬, it turned out that all I needed was a resistor in series for the data input wire. Go figure.</p>
<p>I packed all the wires into a sleek wire cover and stuck it against the wall.</p>
<h3 id="safety">Safety</h3>
<p>With such a massive power supply, I needed to ensure it wouldn’t be on 24/7 so it would be safe and not drive up the electricity bills. I had a <a href="https://www.belkin.com/us/p/P-F7C063/">Wemo Smart Plug</a> lying around that could turn plugs on and off from an app. This allowed me to monitor and control the power supply from within Wemo’s app. I also found a way to switch the plug’s state from my code as well, which gave me more flexibility in controlling the display.</p>
<h2 id="software">Software</h2>
<p>Whenever I work on side projects, my code tends to get really messy and isn’t usually thought out well. This time, I actively attempted to make my code flexible so that it wouldn’t be a pain to reconfigure if I ever came back to this project.</p>
<p><img src="/blog/assets/images/heliohex/schematic.png" alt="Schematic" />
<em>My plan for how everything would work. At the core, a website pings endpoints on a Flask server hosted on a Raspberry Pi.</em></p>
<p>I put together different classes that would allow me to control each hexagon (check out the code <a href="https://github.com/vbhaip/HelioHex">here</a>). I could control each light because I bought individually-addressable LEDs, which allow me to manipulate each LED independently (which is why they cost an appreciable amount more than normal LEDs). To make my project flexible, whenever I would change the arrangement of the hexagons, all I would need to do is change one settings file. I’m only going to explain the most interesting parts of the code (but feel free to check it all out to see how it all comes together).</p>
<h3 id="spotify-syncing">Spotify Syncing</h3>
<p>The coolest feature of this project is easily the ability to sync up to music. The straightforward way to do this would be to have the Raspberry Pi capture any audio, process it with a <a href="https://en.wikipedia.org/wiki/Fast_Fourier_transform">Fast Fourier Transform</a>, and display the lights. Using this method, however, requires a mic for the Raspberry Pi, which is a) more money and b) more electronics.</p>
<p>My solution was to use the Spotify API to extract information about the song playing. This would constrain me to <em>only</em> use Spotify, which was okay considering that was the only application I cared about. I would also have to do <strong>much</strong> more processing because Spotify’s information about songs is more obscure than raw audio waves, in my opinion (which makes sense since someone shouldn’t be able to reconstruct the song from the data Spotify provides).</p>
<p>I started with research. <a href="https://www.youtube.com/watch?v=goUzHd7cTuA">This video</a> helped me understand music theory with Spotify. <a href="https://github.com/yusufsezer/SpotifyMusicLEDs">This code</a> was the only code I could find that was related to my project, but it doesn’t use hexagon lights, just a straight line of LEDs. Regardless, it helped me frame this portion of the project.</p>
<p>Spotify provides its data in discrete information about different segments of each song. After I receive this information, I construct continuous functions for the loudness of the song at any point in time. This gives a loudness value that corresponds to the brightness of the display.</p>
<p>For adjusting the hue of each hexagon, I construct probability distributions on the pitch data for any moment in time, which I sample from to get a hue value for each hexagon. Using a probability distribution introduces some entropy instead of showing the same animation for a song every time.</p>
<p>How do I determine which pitch corresponds to which hue? <a href="https://www.youtube.com/watch?v=JiNKlhspdKg">This talk</a> blew my mind for so many reasons, but the relevant portion shows how pitch can correlate to hue (check out the diagram at <a href="https://youtu.be/JiNKlhspdKg?t=1993">33:13</a> to see the chart with these correlations). I’m not going to describe it here, but the video is 100% worth your time.</p>
<p>The biggest issue I encountered was aligning the display to the exact time of the song. Obviously, if the display isn’t aligned to the song, it’s <strong>very</strong> noticeable, especially when the beat drops. By the time Spotify returns the current time that the song is at, that time for the song has changed. Spotify does return a timestamp when the data was fetched, but that information is <a href="https://github.com/spotify/web-api/issues/1073">incorrect and hasn’t been fixed for two years</a>.</p>
<p>I usually am able to find some hacky solution to these types of problems, but I honestly couldn’t find anything. It still annoys me, but I settled for the unsatisfying solution of estimating the lag at one second which works out alright.</p>
<p>Beyond that, I added some nice features to add biases for the hue depending on the emotions in a song and to add specific color sequences emphasizing the peaks of a song.</p>
<h3 id="web-app-controller">Web App Controller</h3>
<p>I wanted some accessible way to control the light display, so I decided to build a website (much better than an app because I could let guests control the system without them needing to download anything). As with all websites I design, I built a mockup for how I wanted it to look like.</p>
<p><img src="/blog/assets/images/heliohex/controller.png" alt="Controller" />
<em>Mockup for the website controller. The top row shows the original mockup and bottom row shows a revised version. The left column is the display on a laptop and the right column shows a mobile view of the website.</em></p>
<p>I built the website and deployed it to Heroku at <a href="http://heliohex.herokuapp.com">http://heliohex.herokuapp.com</a> (it might take a few seconds to load because I’m using the free version). Each button allows for different “modes” to control the lights. The hexagon display on the screen is generated based off the current layout and lets me change each individual hexagon color (something I’m really proud to have gotten to work!).</p>
<p>To communicate between the web controller and Raspberry Pi, the web controller pings different endpoints on a Flask server running on the Pi, which accordingly changes different colors on the display. The Flask server is on my home network, so you can access the web controller from anywhere, but it’ll only work when connected to my home’s WiFi. The astute may notice that the web controller uses an HTTP header and not an HTTPS header. Because of <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS">CORS</a>, the website must be served on HTTP to ping the local Flask server.</p>
<!--
### Starting on Boot
The last major step was to start the Flask server whenever the Raspberry Pi started. Previously, I controlled the server using SSH, but that was unfeasible because my laptop had to be connected the whole time. I needed the Raspberry Pi to start the Flask script from the getgo.
What I thought was going to be easy turned out to be a bit more complicated. I couldn't just start the script on the startup because the Flask server wouldn't work unless it was connected to the network.
I ended up using [Systemd](https://www.freedesktop.org/software/systemd/man/systemd.html), which allowed me to wait for the network to connect and then run the command. Honestly, it still didn't fully work, so I ended up putting a ten second delay before starting the program to allow everything to get configured, and it worked fine.
-->
<h2 id="final-touches">Final Touches</h2>
<p>The last task for the display was to make it more sleek. I applied clear varnish to the wood to give it a finished look.</p>
<p><img src="/blog/assets/images/heliohex/varnish-comparison.jpg" alt="Varnish Comparison" />
<em>A before and after comparison of the wood without and with varnish.</em></p>
<p>After letting the varnish dry, I needed to combine the hexagons in a way such that it’d be sturdy on the wall, while satisfying my goal of it being transportable. My initial idea was to put each hexagon on push pins against the wall.</p>
<p><img src="/blog/assets/images/heliohex/initial-wall-setup.jpg" alt="Initial Wall Setup" />
<em>My initial setup for the display. This was done before the acrylic was put onto each unit. Each hexagon rests on two push pins, but the hexagons themselves are not connected beyond wiring.</em></p>
<p>This was a bad idea. I desperately wanted to get something to quickly work that I disregarded any real planning. It worked for the moment, but once I touched one of the hexagons, it all came crashing down. The impact put stress onto the soldering points and I had to resolder many connections.</p>
<p>The more thought out solution was to push pin the hexagons together so that they collectively function as one item, providing a temporary yet sturdy solution. Then I could put this display on push pins against the wall. The stability of this structure was so much better. It wasn’t perfect, but it allowed for modularity while maintaining structural integrity.</p>
<p>The last little touch I added was a QR code right below the display. This QR code led to the web app controller, making the display more easy to use.</p>
<h2 id="reflections">Reflections</h2>
<p>This project drained me. But it was so worth it. First of all, it looks amazing - I don’t think any image or video can act as a substitute for seeing it in-person. Second, I learned so much from this project. I used vim exclusively, which made me appreciate just how powerful it is. I implemented parallel processing everywhere - a necessity to get the display in sync. I (sorta) learned about electronics and am not as clueless anymore. And so much more. I improved my skills on technologies I was iffy about before - Flask, Heroku, git, Systemd, and so many more.</p>
<p>Looking back on my goals, I think I did a pretty good job. On the non-negotiables, I’m pleased with everything, though the modularity of the system is not amazing since it is not super easy to switch up the design. I also wished that anyone could control the music for the display, but for now, it only works with my account (changing it would require me to find ways to deal with Spotify’s two-factor authentication and I’ll leave that for another day). (<strong>EDIT January 21, 2021: I actually spent some time fixing this so anyone can use their Spotify account. Now nothing depends on me!</strong>)</p>
<p>Quick shoutout to my family and friends who helped me along the process - having others provide feedback on my progress motivated me to keep going.</p>
<p>There was so much I wanted to write, but I tried keeping it to major and interesting points. If you are interested or have more questions, reach out to me at contact at vinaybhaip.com.</p>
<p>Please consider following this blog if this post interested you. Until my next project, see ya!</p>