These are very rough notes on how I got AWS Xray working in a kubernetes deployment. Posting this mainly for my auxiliary brain.

First, set up an IAM role that allows nodes to access xray.

Following this blog post: https://aws.amazon.com/blogs/compute/application-tracing-on-kubernetes-with-aws-x-ray/

I cloned this repo: https://github.com/aws-samples/aws-xray-kubernetes

Build the AWS Xray image:

cd xray-daemon/
docker build -t xray-daemon:latest .

THIS OUT OF DATE! try this: https://docs.aws.amazon.com/xray/latest/devguide/xray-daemon-ecs.html#xray-daemon-ecs-image

Create an AWS ECR repo:

aws ecr create-repository --repository-name xray-daemon

Login via docker:

aws ecr get-login --no-include-email

That command spits out a command which you can run that will log you into docker ^

Tag and push the image there:

docker tag xray-daemon:latest AWS-ACCOUNT-ID.dkr.ecr.us-east-1.amazonaws.com/xray-daemon:latest
docker push AWS-ACCOUNT-ID.dkr.ecr.us-east-1.amazonaws.com/xray-daemon:latest

You should now be able to see the image in the AWS ECR console

Update aws-xray-kubernetes/xray-daemon/xray-k8s-daemonset.yaml:

Change the image value to the image you just pushed

deploy it:

kubectl apply -f xray-k8s-daemonset.yaml

At school, Kavi is reading “We have always lived in the castle”, by Shirley Jackson. I read her short story, “The Lottery”, in high school. I recall the main plot of the story, but I’m sure I have forgotten many details. What I haven’t forgotten is the sense of shock I felt as the plot finally became clear. As I’m typing this, I wonder if that was more because I was young and new to being shocked, or whether it was really shocking. In any case, that feeling of being shocked by that story is deeply embedded inside me.

So I checked this book from the library and finished it today. It was dark and disturbing, reminiscent of “The Lottery”. There were many elements of foreshadowing. From the start, you wonder what Mary Katherine or her family did to sow such discontent among the villagers, and the answer comes slowly, in bits and pieces. I wondered why they had library books that they would never be able to return. In the end, I was sucked in and couldn’t put the book down until I figured it all out. Thankfully, it’s a short book.

Some random thoughts that my brain generated while reading this book (and that I’d prefer to remember someday):

• When Merrikat and Connie first enter their house after it was trashed by the village, Merrikat is not surprised that the library books are intact because it’s against the law to damage library property. I smiled.
• The absurdity of Merrikat’s behavior at times struck me hard. The first time was when she threw a glass pitcher on the floor because she was upset by a visitor talking to her sister too much. I had to re-read that section a few times because it was written in an almost off-hand way, as if this was normal behavior. She continues to behave like that in bursts, and then the more absurd part becomes Connie’s calm reaction to these behaviors. Piece by piece, you get to understand more about how disturbed they are, and why.
• Merrikat’s belief in magical powers that keep her safe are both hard to believe and comforting. I kept waiting for someone to say one of her three magic words, as if that was the only way that anything could go wrong.
• I wanted Charles to be taught a lesson. I guess it happened, but a part of me wanted it to be more dramatic and more crushing. I’m a sucker for an evil character doing clearly evil things and then getting caught red-handed and punished. I know that’s simplistic, but I still like it and am a little disappointed when I don’t get that satisfaction.

I love the fact that I’m finding books to enjoy, thanks to my kids. Life is good.

I just finished Open Borders, a graphic novel about immigration policy. (Not 2 things that normally occupy the same sentence)

It was an excellent presentation of a position that I’ve always felt to be morally correct, in my heart of hearts. I am one of the few lucky ones who gets to be an American citizen just because of the fact that my parents took it on themselves to forge a better life for us. Why shouldn’t everyone have this opportunity? This book shatters many of the myths that allowing open borders makes things “worse” in any way for natives.

Whenever I come across a book that I want to read, I add it to my Goodreads to-read list. This list, of course, is always growing. I’m very slowly coming to terms with the fact that I will never make it through this list and that someday I will die, with unread books on my to-read list. Horrors! The other major problem with this simple approach to a reading list is that I can never remember why I added a book to my list. If it was a recent addition, I might recall, but in most cases I just can’t remember why I wanted to read the book in the first place.

In that way, the book Deep Survival made it to my to-read list last year, and I picked it up for my Kindle when it went on sale recently:

Knocking off a to-read

I’ve gotten through the first few chapters so far. I have been enjoying it, but something about it is bothering me. It felt very familiar. Maybe that is what happened… I read a teaser for the book in a magazine article or something, and that is why I added it to my to-read list last year.

And then yesterday, while I was converting the blog from Octopress to Hugo, I had to fix up some posts, so I ended up looking through a bunch of my old posts, including…. my book review of Deep Survival.

It’s a weird feeling to read a review about a book that I’ve been wanting to read for over a year, written by a 12 year younger version of yourself.

Kavi was looking over my shoulder as I tried to get my blog working in a local Docker container. I’ve gotten to the point that the site seems to be up and running in the container, but I can’t seem to get to it from my laptop. So, he wondered why I was putting port 1313 in my URL bar (“localhost:1313”) and then I showed the results of netstat -ant to prove that something should be listening on that port. And we got the IP address for google.com and then put that in Chrome’s URL bar with :80 at the end of it, and magically, it worked. (Other websites did not work, and I explained that most sites will not allow their sites to be accessed via IP address).

I still haven’t figured out why I can’t access the blog running in my container, but I think Kavi’s learning more about networking than I ever did (and I’m hoping that he’ll be able to debug these issues for me… soon enough)

UPDATE: I figured it out. Hugo’s server was listening on localhost inside the container, when it needed to be listening on 0.0.0.0. Adding that as a --bind parameter fixed the issue (though I had to also add --baseURL and --appendPort=false params, which saddened me.)

I moved my blog from Octopress to Hugo today. I had been slightly unhappy with Octopress for a while now, but mainly I just wanted something new to play with. Hugo is MUCH faster than Octopress was:

vkurup@caktus020:~/dev/kurup.org(master) $ hugo

                   | EN   
+------------------+-----+
  Pages            | 669  
  Paginator pages  |  29  
  Non-page files   |   0  
  Static files     | 426  
  Processed images |   0  
  Aliases          |   1  
  Sitemaps         |   1  
  Cleaned          |   0  

Total in 808 ms

It is nice to generate my entire site in less than a second.

But I still liked the style and theme of my old site, so I was happy to find that someone had already built an Octopress theme for Hugo. I promptly stole his work and customized a few things here and there to make it look (nearly) exactly like my old site did. Yay! Thanks to Parsia for building that theme. I’ve also subscribed to his RSS feed because his site looks pretty interesting.

Moving my content wasn’t easy, but it wasn’t hard. I mainly just copied the old markdown posts over, changed the front matter, and then fixed a few octopress plugins I was using to work with Hugo.

Future steps:

• Switch from Disqus to something self-hosted
• Generate the HTML on the server rather than having the HTML in git
• Update my About page and Computers page
• … oh, maybe write more!

“Here it comes!” His eyes light up.
Slurps one last sip from his small cup.

Then bounding from the kitchen table,
Fast as lighting, he is able

To trash his scraps and stow his dishes.
He yells to Mom, “It was delicious!”

One more breath then on the couch he lands.
“What’s up?” says Mom. Dad shrugs his hands.

Big brother also has no idea.
Big sister sighs, “He’s a weird one I fear.”

We gather round to be sure he’s fine.
Eyes glued to his wrist, “It’s 7:29!”

And then the beeping fills the room.
The smile confirms he’s over the moon.

The source of this happiness, on his arm,
His brand new watch, and a 7:30 alarm.

(A real-life story, barely embellished)

A kid with a watch

When we made our 2018 goals last year, Kavi, Anika and I all had some running-related goals. I don’t remember offhand exactly what each of our goals were, but I think we all wanted to run more races, because well, races are fun! I somehow stumbled on the Triangle Race Series. It’s a series of a couple dozen races throughout the year around the Triangle, so we signed up for a few. They keep standings based on participation and placement. You get 5 points for running a race, and more if you place in your age/gender division. We ran about 4 or 5 of the races. I hadn’t really thought about the series at all recently since I injured my right Achilles in July and haven’t run at all since then. Getting old really is a bummer. Anyways, I checked the website today and they have posted the final 2018 results. I got 6th place in my “not-quite-old-fogey” division! Not bad, huh? Yeah, I know… it’s mostly a participation award, but still, I’m proud. But wait! That’s not even the best part. I’m even more proud of Kavi and Anika. They are both in the 0-19 age group, so get to compete against teenagers. Kavi got 3rd place in the male 0-19 division, and Anika got 1st place in the female 0-19 division!

Oak City Mile 2018

I got 2 Nest E thermostats because Duke Energy was having a sale around Black Friday. I didn’t have any strong motivation to have a Nest. The idea of a smart home is cool, but most ‘smart’ things nowadays have ended up being more work than benefit. Anyway, for $75 each, I thought it was worth a try. We have 2 thermostats so I decided to install the upstairs thermostat first. The process started off smooth and easy. I like that they provide a back plate, which I didn’t even notice until my trusty assistant, Kavi, pointed it out. My existing Honeywell thermostat is rectangular and if I just put the small circular Nest in its place, there would be a big chunk of unpainted real estate on the wall. Having the back plate prevented this from being a ‘replace thermostat, and learn how to patch and paint the wall’ project. The online Nest compatibility checker and the paper instruction manual (which includes stickers to label the existing wires) are also nice touches. That all gave me the (misguided?) confidence that even I could do this. A study confidence which lasted until I flipped the circuit breaker back on and …. nothing happened. According to the manual, the Nest was supposed to turn on, say Hi, and then walk me through the set up process. But it remained dead as a doorknob. I pulled it off the wall and plugged it in via USB (a nice troubleshooting feauture). It turned on and then when I placed it back on the wall, it complained that it was not getting enough power from the R wire. The error code was E195. When I looked at the ‘Technical Info’, it reported that the R wire was only providing 2-3 volts, while it needed 24. I chatted up Nest support, which took about 30 minutes to connect on a Friday night. The representative was responsive and useful. They eventually recommended trying to disconnect the C (common) wire. For a reason I still can’t explain, this brought the voltage on the R wire from 2-3 volts up to 35 volts, and my Nest was working.

Having tasted the sweet nectar of success, I then tried to swap my second Nest in for my downstairs thermostat. I made a huge mistake here. It had been easy to identify the circuit breaker for my upstairs furnance because it was labelled, “Up Furnace”. But there was nothing labelling the downstairs furnace. I switched off the circuit breaker that I thought was responsible for the downstairs HVAC, but in hindsight, I think I got the wrong one (and since it was warm out and the heater wasn’t running, I didn’t notice because nothing was running in the first place). Anyway, I walked through the rest of the instructions and again got to the point where the Nest was supposed to power on, but it didn’t. So I powered it on via USB and placed it back on the wall. Unfortunately, this time I was getting only 0.01 volts from the R line. Still, I was hoping that the ‘disconnect the C wire’ trick would work again. Unfortunately, disconnecting the C wire did nothing. I walked through things with Customer Service chat. Again, they were helpful (rather than just spouting cookbook-type basic info), but in the end, they didn’t have a recommendation that would fix things. They suggested that I get a voltmeter and test the voltage output at the various wires. I do not have a voltmeter and felt that I was getting in over my head, so instead I put in some calls to HVAC service companies, but since it was the weekend and this wasn’t urgent, I was resigned to having no heat over the weekend. I watched a few Youtube videos of HVAC repairs, some of which were quite entertaining. Eventually, I tried to reconnect my old Honeywell thermostat and was surprised and intrigued when this ALSO failed. It made me think that I messed something up somehow. The only thing I could think of was that since I hit the wrong circuit breaker, there was live current in the wires. While moving them from the Honeywell to the Nest, perhaps I touched 2 wires together causing the system to short circuit. I don’t remember noticing a spark or anything, but that was my best guess. I eventually watched enough videos to give myself confidence that I could open up the HVAC and find the electrical panel. I did that successfully (after turning off the CORRECT circuit breaker, and also pulling out the pull out switch next to the furnace). After cleaning out a ton of cobwebs, I found the LED light on the electrical panel was off (even after connecting the circuits temporarily). I searched the panel and found a 3-amp E fuse (like the kind I’ve seen in automobile fuseboxes). I pulled it out and sure enough, it was blown. It looked like the ‘bad fuse’ in this photo. Off to Home Depot, I purchased 2 replacements for 3 dollars, and then replaced the busted fuse with the new one. I flipped the circuit breaker and put the pull-in switch back in. The first encouraging sign was that the red LED light on the electrical panel turned on. I went inside and, joy of joys, the Nest was booting up and showed 35V via the R wire. So, I guess the bottom line is that I need to be more careful when I’m at the circuit breaker box.

Guess what, we’re composting now! I told you was going to be a gardener, so now you know I’m serious. For oh so many years, every Friday, we’d haul the garbage and recycling out and I’d think to myself, “Wow, we’d have like no garbage at all if we were composting.” It’s only taken a few years of thinking that, week in and week out, to get me to try. Actually, that’s not even true. It was still way down on my future to-do list until I got a cheap copy of “The Complete Idiot’s Guide to Composting”. Thank you, BookBub! (As if I needed any other incentive to buy and read more books). But honestly, the book was excellent in that it made me think that yes, even I could compost. So shortly after finishing it, I headed down to the Orange County Solid Waste Center and purchased an Earth Machine compost bin for fifty bucks, and made the kids set it up near the screen porch. We’ve been throwing all our copious kitchen scraps for the past 2 weeks. So far, all that has happened is that a bunch of fruit flies think we’re awesome neighbors and seem to have moved in for the long haul. But I’m hoping that some composting starts happening soon. It’s supposed to be easy as pie, but I’m secretly worried that I’m missing something, oh, like maybe skills or talent. But our garbage bin sure is lighter!