Browsing Category

Blog

I Survived a Sim Hijack

Posted on May 11, 2020

About 2 months ago, I was leaving a restaurant in LA after dinner when I noticed that my cell phone had no signal, so I restarted the phone (which normally fixed it) and was greeted with the following text message.

I immediately knew someone was either in the process of breaking or have already broken into one of my accounts, so I quickly sprinted across the parking lot, jumped in my jeep, dialed T-Mobile, and slammed on the gas. Somehow, I already had cell service back on my phone. In a panicked call as I was careening down the 405, there was some confusion between a pin and a password with the support rep and they kept asking for my password. It wasn’t until later that I find out they should have asked for my pin, I was never going to give someone my password and you shouldn’t either. T-Mobile accounts also have a PIN which is the number you give them as a private key. This was the first time I ever needed support and wasn’t familiar and the support rep wasn’t explaining so the call ended with them just recommending I visit a store in person the next day, because it was 9:30pm. Thanks for nothing.

When I got home 15 minutes later, I quickly checked my phone and saw the damage.

They got me. The bastards got me, Clint.

It’s weird, seeing one of your accounts compromised like that. There’s a first time for everything I guess. Coinbase? I barely even remember having an account on there, must have been at least 5 years old. I didn’t have anything in the accounts last I remember, that’s why it went dormant, but then I remember that my checking account is linked to it.

As I approached my front door, I recalled seeing something in the news regarding malware and Coinbase so I quickly googled ‘Coinbase sim hijack’ and was greeted with this headline.

Oh…Damn.

If it’s embarrassing for a crypto engineer, how embarrassing is it for someone in security? Really, this is a very embarrassing story for me to admit to, but there is a lesson in here so let’s keep going and try to have a good time.

I practically kicked the door to my apartment like SWAT and ran to my desk and validated that yes, I was locked out of my email account. It was a little surreal to be honest, but I had no time to get all existential and shit, I had an active breach I needed to resolve.

Currently, I was in step 3 of the Incident Response Life cycle, Containment, Eradication, & Recovery. Which also works as a sick band name.

From NIST’s Computer Security Incident Handling Guide

In that moment, I didn’t know the full extent of the compromised accounts or even how they got in. I quickly saw news results of malware sniffing some password manager passwords and to be safe, I started kicking off virus scans of everything, I reset my main password and some of the bigger ones connected to my finances because I wasn’t sure and wanted to be safe.

My phone had service again because T-mobile was able to detect the SIM change as fraud and had already swapped my number back before I got in my Jeep, so I was able to utilize the 2FA SMS settings on my Yahoo account to log back into my email and change my account password back and then try and login to Coinbase, which was an an account I experimented with, but haven’t used in years. Eerily mirroring this story I saw of a guy who got $25k taken out of his checking account with the line, “Coinbase account I experimented with in early 2018 that was never closed”. I didn’t have my Coinbase credentials in my password manager because the account went dormant before I even started utilizing a password manager, so it was an unknown risk. But it sure as hell wasn’t unknown anymore.

I reset my password on Coinbase and I successfully login to see this.

Either the attackers just turned on 2FA to further add another hoop I have to jump through or I had an old 2 Factor account I don’t remember. So I then file a claim with Coinbase and have to send a photo of myself giving my webcam a resting bitch face and my ID to recover the account and it might take up to 48 hours so here’s to a nice relaxing 48 hours.

I called my bank and alerted them and they locked down what they could, but at the time, I wasn’t sure if it was still too late to recover any money taken from my account. At the very least, the 2 accounts compromised were recovered. It’s 2am, I should probably get some sleep, but I had to keep digging. I went through old emails and checked names of old authenticator apps and found that I did actually have an old account to one. The adrenaline high I was on didn’t allow me to go to sleep until about 5am.

After a nice “relaxing day filled with zero anxiety whatsoever”, I finally received an email in the evening informing me that my Coinbase account is recovered and I login to see this.

Oh neat, talk about the least chill way to find 50 bucks.

So it appears that back in 2013, when I setup my account, I did so with non SMS based 2 Factor Authentication. That was the control that prevented the attacker from logging into my account in the end. If I would have felt lazy that day and said I would do it later and never did, they would have been able to withdraw essentially all the money I had in my PNC Checking and Savings account. But now instead, I actually discovered money I thought I didn’t have and thus made money from this breach? That’s not exactly driving home the lesson I’m trying to give here though. I could have lost practically everything.

How Did This Happen?

I first looked into T-mobile since the issue started with the phone, I saw a breach very recently in the news and thought they may have swapped the SIM with my account creds until I realized that the T-mobile account isn’t associated with that yahoo address at all so that means that even if they did breach that T mobile account first, they never would’ve know to use it with my yahoo email.

I know what you’re probably thinking, Logan, why the hell do you use Yahoo Mail? Especially since the FBI or NSA had a backdoor years ago? It’s a free email account that I can use for anything spammy or craigslisty. I moved everything important to a private domain email years ago, or at least I thought I did.

Like Gandalf hauling ass to Minus Tirith in the beginning of Fellowship, I quickly ventured to HaveIBeenPwned.com to check my email and bingo.

This yahoo account was in a couple of different breaches before I moved to aliases and I’ve changed my password numerous times, but this is the first time it was in a breach in combination with my phone number. That reminded me, so I went digging and about week earlier I received this email late on March 1st.

At the time, I quickly changed my password to an extra long hashed password this time and ensured that I had some sort of 2 factor enabled, but there was a flaw. Despite those 2 factor settings, my Yahoo settings allowed me to recover my account via SMS, and that made me vulnerable to a SIM hijack. Adding to it, using the same SMS number for 2 factor authentication nullified the entire point of having a separate 2nd factor in this case.

At the end of the day the accountability lies with myself, but at the same time, I feel like Yahoo should warn users if those settings are selected. To show just how quick it can be, if I click on “Forgot password” with my email in the text box, I’m asked if I have my phone and it shows my partial phone number. I just have to confirm with a single button and then I get a text message.

Entering the 6 letters correctly like a Zelda puzzle is all someone needed to do to change my password and walk right into my yahoo account.

So if I want to login to Yahoo, I’ll need my email, password, and my phone. But with Yahoo’s SMS Recovery feature from hell, if an attacker wants to login, they just need my email and phone number to swap the sim

If I login to Yahoo traditionally with a password from a new device, I’ll get an email indicating that, but since I only was informed of changed password, I assume this is the method that was utilized.

When I finally talked to another T-mobile support representative later, I asked if they could see the history and what the previous activity was. They said it looks like the sim hijack was done what they said ‘through the system the stores are connected to’, but they didn’t say where the location was, but seemed as confused as if it wasn’t something they normally saw.

The problem with my thinking about the risk of a SIM hijack was generalizing the probability within the risk. My thinking was since I wasn’t a celebrity and I’m definitely not high enough on the totem pole of being a whale, the probability has to be what, 1 in n where n is the size of T-mobile customer base? That type of assumption is wrong because it assumes that there is only going to be 1 SIM hijack for the entire T-Mobile customer base.

You cannot properly calculate risk without context.

I don’t know the exact method the attackers used, but if I were to imagine someone with the ability of changing a sim having access to a list of Coinbase usernames, all you would need to do filter by ‘@yahoo.com’. Then from that list, try logging to every single one of them to see which ones are vulnerable to a SMS recovery.

You could easily automate this and within minutes probably have a list of Coinbase accounts associated to a vulnerable Yahoo account with their phone numbers. All that would be left is swapping the sim.

The regular routine of password changes I made to my Yahoo account did not resolve the overall vulnerability that persisted due to the configuration of both my Yahoo and T-Mobile accounts. That combined with an ocean of various attackers farming through multiple lists of emails, it seems more like it was actually inevitable and only a matter of time. This is why probability should be viewed less as a single static generalized number and more as one that increases over time without treatment.

My saving grace was utilizing proper 2 factor authentication with Coinbase. In contrast to how Yahoo operated in this case, Coinbase allowed a password change, but not a login without the 2nd factor. They could have changed my password a hundred times, but it would’ve mattered without the 2nd factor which wasn’t connected to my phone number.

Coinbase has a pretty nice account activity page and you can see from the bottom up, the activity leading up to my password being changed starting about a month ago.

This screenshot was taken 5 days after the compromise. My activity is removed. You can see there were numerous attempts on my account via the api, a couple days later was when I had that attempt into my Yahoo account, I believe that may have been to check to see if my account was setup for SMS recovery. It took about 5 days to finally swap the SIM card in the end, like this is a weekend gig for this piece of shit.

Lessons Learned

In the post incident phase, you need to take a moment to document the exact things you did wrong or didn’t do, things that you definitely need to fix in the future, and the things you did right, things that you want to increase to other areas if possible. First off, what were the things I did wrong?

1.) Not having a proper inventory of my accounts

You cannot protect protect what you don’t know. I did not have a proper inventory of all of my accounts in a single place until I got a password manager, but even when I did, I didn’t properly go back all the way into my email and find more dormant accounts. Especially accounts that are connected to my checking account. What the hell, Logan?

Also, consider this your daily reminder to get a password manager if you don’t have one. They also can also help assist you with creating that inventory of accounts.

Proper inventory in my opinion includes regular checks for breaches. Even though I have checked HaveIBeenPwned.com before, I haven’t checked in the last year or so when my phone number included in a breach with my email. Being aware of that might have been something that would have pushed me to remediate the next down the line issue.

One more thing about proper inventory is proper decommissioning, if I know I’m not going to use something, I should remember to delete the account rather than just delete the app.

2.) Not validating I was secure from SIM Hijacks

This was one I am really am eating it on. I knew of SIM hijacks, I’ve read stories on them, but I just kept kicking the can down the road for myself when it came to ensuring I was secure. I was always too busy. If you’re reading this and you haven’t validated that your SIM is safe with the instructions below I heavily suggest you do so.

For AT&T

AT&T allows a passcode to be added the account which would be prompted in situations like sim change. Their guide is here.

For Verizon

Go to VZW.com/PIN to set your PIN. As an alternative, you can stop by any Verizon store with your government-issued ID or call 1-800-922-0204.

For T-Mobile

To do this, you’ll need to call 611 from your T-Mobile phone or dial 1-800-937-8997 from any phone. The T-Mobile customer care representative will ask you to create a 6-to-15-digit passcode (or called 2nd PIN) that will be added to your account. I’ve also heard there is a way you can request that any bigger changes such as SIMs be made only in person at stores. However, it still leaves it up to T-Mobile to follow that.

For Sprint

Sprint requires its customers to have a PIN. It’s a good idea to periodically change yours by logging into your account. Once you’re in, select “My Sprint” and then “Profile,” followed by “Security.” Scroll down to “Security Information” to update your PIN.

For other carriers, just google them and sim hijack to find your solution.

I have heard of some people’s preferred solution was to get a Google Voice number for all of your SMS 2 factors, but I’m not really sure about tying more of my life into Google, so it’s up to everyone’s comfort level.

3.) Using SMS 2 Factor

Some may view this as steep, because some websites only give the option for SMS 2 Factor, but even if you follow the hardening instructions in the previous section, some articles have documented people using T-Mobile experiencing a sim hijack despite making the call to their carrier. So I don’t want to say that the call is meaningless, you should absolutely make the call if you haven’t but not knowing all the intricacies of the entire wireless system and how the system processes a SIM change request at a much deeper technical level, I would say it might not be 100%.

Researchers from Princeton University analyzed five prepaid wireless carriers’ authentication procedures to understand how they processed requests to change SIM cards. They signed up for around 50 lines across AT&T, T-Mobile, TracFone, US Mobile, and Verizon and then called to request a SIM swap on each account.

“We found that all five carriers used insecure authentication challenges that could be easily subverted by attackers,” they said. “We also found that attackers generally only needed to target the most vulnerable authentication challenges, because the rest could be bypassed.”

Danny Bradbury has a more in depth summary here and the full report is here. At the end of the day you should know that as of 2020, wireless carriers in the US aren’t treating this as seriously as they should. SMS 2 Factor shouldn’t be used unless where there is no other option. NIST depreciated SMS as a 2 Factor method in 2016, however using it as a 2nd factor is still better than not using any other 2nd factor at all.

So what did I do right?

1.) Using non SMS 2FA

I’m really beating a dead horse here, but if I didn’t use that separate 2nd factor for Coinbase, or if I would have used SMS 2 factor, they would have just gotten another text message and would have walked right in.

If you’re not sure of what to use, some password managers come included with an authenticator. Here’s a list I googled and am not affiliated with at all.

This was such a scare, I’m lucky my experience wasn’t similar to the horror stories I’ve seen online. When I played college football, I played under Joe Walton and he used to say that “winning covers a multitude of sins”. He meant it that you could have a really shitty game, but at the end of the day if you won, it didn’t really matter as much. I’d say maybe in this context, the right control can sometimes cover a multitude of sins as well.

Is a VPN The Answer?

Posted on May 5, 2020

With COVID-19 keeping everyone inside for the last few weeks, I’ve been getting a lot of questions from friends about VPNs from and if they are worth the investment. My answer has always been the same, “Oh shit, well, umm I will try and write a blog about it.”

Because it’s never that simple, it’s never a yes or no scenario. Everything comes down to the nuances of your specific use case and threat model. What are you doing, what are the risks, and what are you trying to prevent?

I was having a conversation with a friend a few days ago about his home network and he mentioned he was in the middle of buying a router and then flashing the operating system to install DD-WRT, and then he was going to route his entire network through a VPN in another country and then also bring Tor into the mix while at the same time, he was a self confessed novice at networking, This was a “solution” recommended by a security friend.

“What are you trying to do?”, I asked.

“Oh just trying to protect my data, from my ISP selling it, privacy and all you know?”

My reaction would have been the same if my 63 year old mother called me to tell me she’s rebuilding a transmission of her car. I get it, but given her situation and knowledge of the subject matter, it might be better to just look into another solution if she’s just looking to drive.

It’s very common for security marks to recommend moonshot security solutions for people not too tech savvy. Part of it is just a little disconnection from regular people’s use case, part of it is gate keeping bullshit which is another blog post for another time, but when someone lacks an understanding of the back end functionality of the technology and various externalizes, they could find themselves implementing a solution that actually could make them more exposed.

So, if the use case is ensuring your privacy and preventing your web browsing from being “sold off”, do VPNs like NordVPN, PrivateInternetAccess or ExpressVPN do it?

To answer that we have to go over just how internet connections are made.

On a default setup, if I were to type www.pornhub.com into my address bar, my computer would query my ISP’s DNS (Domain Name System) server for the IP address. My ISP would then respond back with the IP address it either had in its cache or it would query another server downstream to get the address. Upon receiving the IP address (66.254.114.41) from the DNS server, my computer that would then initiate a connection to it. This whole back and forth happens totally unencrypted with your ISP. Every single .com, .org, or any other website name address that your computer queries, whether by you manually entering it in the url, or queried by ads, or any other application on your computer is done via plain text and is viewable by your ISP, like below

Utilizing a VPN, my ISP would only see an encrypted connection to the VPN’s server, but the VPN would still be able to see everything my ISP used to see which means that at its core, utilizing a VPN for privacy is just a shift in trust. Now, for many, that shift in trust is a quick simple assessment but the bigger, more centralized problem with VPNs is that they’re being oversold as silver bullet solutions for many unsuspecting consumers and constantly parade around incredible claims. Now, if a VPN has no problem being deceptive with how a VPN operates and their effectiveness to me, why on earth would I believe their claim that they take my privacy seriously when they don’t even take me seriously?

Snake oil and homeotherapy may be too strong of analogies, because VPNs do actually have a legitimate use in some circumstances way they are pushed in online marketing, VPN ads are beginning to rival end time evangelical preachers in terms of their far fetched claims and fear mongering. Go to any VPN site, they probably will say something on that webpage like, “YOU ARE NOT PROTECTED” on their page because your IP isn’t using their service. That’s all those web pages check for. There’s no other magic happening where your computer is scanned and has a hole. It just means, you’re not a customer.

If you recently ponied up a bunch of money for a VPN, you will probably claim that your VPN doesn’t log and I would say your VPN probably has a nice website that says that it doesn’t log, but because neither you or I are able to see firsthand the infrastructure and validate it’s practically meaningless. Many VPNs have to rely on a basic set of logging to enforce device limits and other controls. Even if they do have audits and fancy badges on their site to prove they don’t log, that doesn’t mean that someone else isn’t monitoring the infrastructure those VPNs are on. And before you ask, yes, there have been VPNs like H.M.A that have given up their users before, so why would any others stand up for 1 user in the face of a subpoena?

If you’re not too tech savvy, the reason you’re probably thinking of a VPN is because you saw an advertisement or a promotion that really seemed enticing, or some youtuber or blog included in in a post. If you google ‘VPNs’, half of the results are entire websites dedicated to top 10 reviews of VPNs, that’s literally just an interactive ad. Be aware, despite flashing an impressive perception that a VPN gives off due to marketing or reviews, some companies are involved in shadier practices behind the scenes. While others have been involved with producing malware. So don’t automatically assume a VPN has your best interests at heart just because they come off very honest. Some VPNs were found to deceive and inform the users that they were routing their data through one IP address when actually on the back-end they were being actually being routed through another. Suddenly that initially quick and simple shift in trust isn’t looking so simple.

VPNs can not be assumed to be a fully effective way of hiding your identity either. If you login to the same services while logged into the VPN. Using aggregation from a number of other data points can be used to identify you, I offer up this 2016 criminal complaint in the US District Court for cyberstalking. The defendant had a computer science degree and was well aware of how VPNs operated and he was still able to be identified.

Imagine that, a pitch to be this super private, anonymous service that is the perfect honeypot for those with more “interesting” traffic only to track it all anyway. I’m not a tinfoil hat conspiracy theorist, but the the assertion that many VPNs could be honeypots is a reasonable one. The amount of money spent within the technology media spheres advertising and seeking out those looking for people looking to hide should cause one to at least reflect on the possibility of it.

If hiding from the government is more your thing, if the VPN is headquartered in the Five Eyes or the Fourteen Eyes countries, it’s subject to data retention laws. That is why the U.S. and U.K.-based VPNs are generally not recommended by privacy-protection organizations and security experts.

You could probably find one ran out of a shack in Iceland that really protects your privacy, but then come the questions about reliability, network speed, etc. “I’m so glad I have this gigabit internet connection routed through a 50Mbps VPN in Iceland.“

Creating a VPN isn’t hard. I mean it is for a non technical person, but it’s not as hard as an entire company being dedicated to it would make one believe. If someone had internet access to their meth lab in Barstow, they could technically run a VPN server and sell access from a fancy looking website pumped up by Reddit reviews and we might not even know the difference. Hell, the VPN could be a front run by an adversarial state and we wouldn’t know the difference.

I’m not saying VPNs don’t ever serve a purpose, there are plenty of proper reasons to utilize a VPN and in fact, I’ve previously written about using some of them. My main use of VPNs are to tunnel between devices so I can stream with more stability. My work has a VPN so I can access that network over an encrypted connection. Other legitimate uses are for tunneling through untrusted public networks like starbucks or an airport, or even bypassing government blocks or geolocked content.

I’m also not saying that all VPNs are bad either. There may be some that legitimately don’t log your activity, who’s client isn’t malware, or have security issues, or who’s network isn’t externally monitored themselves, or who’s client doesn’t leak your DNS requests back to your ISP, or who’s infrastructure doesn’t slow down your speed, but it will take a lot of time researching and finding it. Thatoneprivacysite.net probably has the most definitive listing and ranking of them. I know a couple people who opt in spinning up their own on the own server they know doesn’t log, but what about the network those servers sit in? I don’t know if you can find that unicorn solution that hits 100% of those checkboxes and ensure total 100% privacy, because at at the end of the day, its still just a shift in trust. For some, that shift brings with it more cans of worms than not.

Everyone has their level of risk tolerance, or the amount of risk they’re willing to be exposed to (or own) before they take an action to remediate it if at all. Some aren’t comfortable with their ISP and to them and their situation, it makes sense. Others, might not.

Now, I write from the United States and the reputation of ISPs here is far worse than other places. Many of the things ISPs do here with customer data is considered illegal in other countries. However, at the same time, the amount of censorship that’s other countries experience which warrants a VPN is far worse. I would say that US ISPs still have more to lose than a random VPN that could be run by some college kids in Serbia with a nicely designed logo. If you’re worried about ISP specific DNS Blocks you can update those to alternative ones, but remember, that’s sharing those unencrypted DNS requests with those new DNS revolvers now.

I know what you’re probably asking in your head, isn’t there something I can do that’s more impactful than simply changing my IP via a VPN that would help protect my data?

Sure, but unfortunately it’s not as fun and exciting installing an application with a big fancy padlock that says ON when you start it. It’s actually a little time intensive, but if you live in California like I do, you can legally request ISPs and other companies not to sell your data to marketers and you should track down every single CCPA portal for every company you know that has a data profile on you and you should opt out.

I know these are mostly effective because the amount resources the previous companies I’ve been employed at spent to comply with this and GDPR were extensive because the penalties for not complying are real.

For example, here’s Spectrum’s opt out website for CA residents.

Here’s Frontier’s, which is a little more of a hoop to jump through.

T-Mobile’s here, you should google the company name and “opt out ccpa” to try and find any company you know that might have a data profile on you.

What if I’m outside of California? Check with the laws in your local jurisdiction for the privacy implications and data collection laws. Definitely look into at least encrypting your DNS requests with DNS over HTTPS or DNS over TLS. Install uBlock on your browser or look into setting up a Pi-Hole which either would actually be more impactful by killing tracker requests than a VPN which would just route those same requests from another IP.

Furthermore, many who ask the VPN question do so before taking more meaningful steps to protecting their privacy and securing their data such as assessing their data footprint or account vulnerabilities with a password manager. Or going down and turning on 2FA on all the applicable accounts, or ensure all your programs and browsers are patched and updated.

At the end of the day, is a VPN the answer? Well, it still depends, I mean if the question is a tunneling one, then probably. If the question is looking for a simple privacy solution or to be a silver bullet for your entire risk portfolio that many VPNs are guilty of masquerading as then you are going to be disappointed.

Further Reading:
https://www.reddit.com/r/VPN/wiki/beware_of_false_reviews
https://krebsonsecurity.com/2017/03/post-fcc-privacy-rules-should-you-vpn/
https://gist.github.com/joepie91/5a9909939e6ce7d09e29
https://www.engadget.com/2017-04-07-good-luck-finding-a-safe-vpn.html
https://drewdevault.com/2019/04/19/Your-VPN-is-a-serious-choice.html
https://www.natlawreview.com/article/ccpa-qotd-what-are-penalties-non-compliance-ccpa
https://www.makeuseof.com/tag/avoid-bad-vpns/
https://www.techdirt.com/articles/20170808/00222037952/complaint-filed-over-sketchy-vpn-service.shtml
https://schub.wtf/blog/2019/04/08/very-precarious-narrative.html

Remote Playing PC Games Part 2: Playing Outside Of Your Home

Posted on July 31, 2019

Ok so I can finally play my PC games while laying on my couch in the other room, that’s neat. However, despite the fact that playing anything on a bike at the a gym makes me feel like a total POS, I want to see if I can play my PC games from there.

Anyone want to check out this sick goal I just made in rocket league? Ok, whatever.

From some initial tests, I was getting good connections, but it would always cut out and terminate the connection to the point where every place, even my work’s fast internet wasn’t playable. It wasn’t until I tried utilizing Zero Tier as recommended on Moonlight’s troubleshooting page that I really found success.

Zero Tier

Zero Tier is a Peer to Peer VPN. It will allow you to connect two devices together such as your phone or streaming PC. Following the instructions online and installing Zero Tier on my phone and laptop has all of our devices connected each other within 10 minutes.

I don’t want to spend any time getting into Zero Tier, but the only change I had to make is that once the new network is added to my devices, I had to re add the computers in moonlight (and re-pairing the device to the streaming PC) by their new virtual IP.

Just to give you and idea of the speed of my PC to my ISP,

Be cool to your cable guy, he might give you a little more megabit than you pay for.

This is the bandwidth to the Frontier server, but since it’ll take some hops it’ll be better to run iperf tests in this instance.

iPerf is a widely used tool for network performance measurement and tuning that can produce various performance measurements for any network. We’ll have it running on the streaming PC and then when measured from the client, it’ll measure it directly, rather than only to whatever Ookla’s various servers on speedtest.net.

iPerf3 is also available on Google Play so we can measure our speed from a phone too.

24 Hour Fitness
1 World Trade Center #110, Long Beach, CA
Network: T-Mobile LTE
Platform: Samsung Galaxy S10e
Distance: <1 Mile

Hats off to the couple of users who did a speed test in boats in the bay.
iperf results shows 23Mbps between my phone and my streaming PC

I felt like this was going to be a “gimmi”, because i’m only a few blocks away from my apartment.

I recorded this video before I installed Zero Tier. This was one of the few use cases that worked without it. Also, as a side note, 4G LTE here is hilariously faster than the 24 hour WiFi network, so I didn’t even bother with the wifi here.

Santa Monica Business Park Parking Lot
3010 Ocean Park Blvd, Santa Monica, CA
Network: T-Mobile LTE
Platform: Samsung Galaxy S10e
Distance: 28 Miles

Sometimes traffic is so bad on the 405 it might be better to chill in my car for an hour before heading out.

T Mobile LTE map for Santa Monica. Notice all of the little ‘v’ symbols stand for “Validated” meaning the speeds are collected from users who agreed to share diagnostic data.

Right now we’re in the top tier coverage area for T-Mobile’s 4g LTE coverage, so let’s rip it.

Iperf test shows around 11.5Mbps which should allow us to play at a lower bitrate.

This is absolutely playable. I don’t have any external footage to show the latency and there is some delay, but it’s well within playability. I feel confident I could fully play the witcher 3 fully in this setup.

This actually works, wow. I know I shouldn’t be that surprised at this point with technology, but I am.

So how much data is this using? With wireshark, I was able to measure the network activity of a couple different settings and as my Grandfather would say, “Good Golly Miss Molly.”

Playing at 30Mbps at 720P will still take up around 500MB in only 222 seconds. Playing at 6Mbps isn’t much better at 900 seconds(15min). So if you play this for an hour at lower settings around 6Mbps, you will use up 2GB of your data plan. Yowzers.

Allow me to present my graph comparing 30Mbps and 50Mbps steams.

It’s not a 1:1 accurate comparison in the wireshark analysis because the video being streamed was slightly different and different amount of movement on screen will translate to the codec to compress at different rates, but i think it still gives conveys a general idea not to remote play over LTE unless you have an unlimited plan. Unless you’re bougie.

Grandma and Grandpa’s house
Scottdale, Pennsylvania
Network: 802.11ac 5Ghz
Platform: Dell XPS 15 2-in-1 9575
Distance: 2,455 miles away

Let’s really see what the US infrastructure can do!

Next time you’re visiting the in laws at thanksgiving, give em the ol’ “I ate too much bird” excuse and run upstairs and get your PJs on, we gon’ play.

Now I know what you’re thinking, if it’s THAT important to me play games remotely, I should be fine justifying the purchase of a much more expensive gaming laptop and you would be right, but still, I wanna see if this is possible.

Not bad for rural Pennsylvania.
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  13.8 MBytes  11.5 Mbits/sec               sender
[  4]   0.00-10.00  sec  13.6 MBytes  11.4 Mbits/sec               receiver
iperf Done.

This should be fast enough if we keep the video bitrate to around 5Mbps @ 720P.

I recorded the video with windows embedded screen recorder and then uploaded at 1080P 60fps and Youtube’s algorithm blurs a little more of the already compressed areas. The video is sharp when things are still, but once anything moves it gets very blurry. Technically, however, it is playable. The latency is a little more noticeable here, but not as much as I was expecting beforehand.

Technically, I could sit here and play this, but at this quality, I would rather just wait until I’m sitting at home in front of my TV, but within the next few years or so, I don’t see that continuing to be a problem.

Conclusion

I’ve seen a lot of people argue that Google Stradia and other game streaming services are going to fail, but I think this little exercise showed me that it’s a legitimate next step. The next step.

Everyone else’s servers are going to be far more powerful than my rinky dink PC, and they will have those servers located far closer to everyone than 2500 Miles. With the adoption of 5G, I think it’s perfectly reasonable that conceptually, the line separating PC Gaming and Mobile Gaming will blur to the point that the idea of mobility being this distinct, defining attribute of a sub group of gaming will no longer apply and that those specific mobile attributes will be available for all games. Mobile games will no longer be a thing because all games will be inherently mobile.

Remote Playing PC Games

Posted on June 25, 2019

Update April 10th 2020. I changed the name of this blog to more clearly reflect the terminology shift and to not confuse Remote Playing with Streaming Games on Twitch. I have not altered the text or the url of this blog, but in this context game streaming is essentially Remote Playing.

Note: Because everyone’s network is different, The best way to be 100% sure of how Moonlight runs on your network is to try it out on yours. This post is not a technical deep dive, but more of a brief overview with a couple quick examples from within my local area network. The next part is going to go a little more in depth in requirements to stream outside of a LAN and some attempts to stream over different connections.

When I was in high school, circa late 00s, laptops just hit the point where gaming on them was just getting to be a tolerable use case, but not before dropping some serious cash. I saved up for over a year to finally buy a 17 inch Macbook Pro that can boot windows and play some good beefy games, it was the best laptop I ever owned, but it was $2900.

$2900? That’s more than my car.

After college I went back to a desktop and haven’t needed a mobile solution until recently now that I’m out of the house far more often. However, I don’t want to drop more than a grand to play something at high settings. I already have a Ryzen 2700 and GTX 1080 at home that can run almost everything. Is it possible to stream it? Years ago I tried, but I was just on the cusp of it.

Back in 2013, I was one of the few who jumped onto the Ouya bandwagon. It was an android based TV unit that came with a wireless controller. So many different things could be done to it. I mainly used it as a Plex client on my TV for years until I upgraded to an Nvidia shield in 2017, which is practically the same thing, but with a faster CPU. Back then, there was an application called Kainy that allows a user to stream video from PC to the device play games from that client TV. I made a video testing out the performance, it was “ok” at the very best.

It wasn’t optimized, games would run windowed or there would be a half second lag over Ethernet. It was nice to show as a concept, but it wasn’t enough to replace my gaming setup. And back in 2013, very few PC games integrated with a controller as easily. I still made a video, because at the time there was barely any results or documentation on it.

Barely playable over Lan on a few games.

It’s been some years since, so how has performance changed? Some of the big players are coming out of the woodwork and pitching their flags. Google Stradia, Shadow, Bethesda Orion, Microsoft xCloud, judging by the number of players in the game, one has to assume the tech and speeds have caught up, but some services like Shadow cost $24.99 a month to remote into a virtualized desktop running in a datacenter.

I already have the computer than can run the games I want to play, I don’t want to involve a datacenter. I recently bit the bullet and upgraded to 500Mbps up and down and 1 Gigabit on everything Ethernet within the LAN so I should have all the bandwidth necessary. All I need to do is find the right program and that was when I stumbled onto Moonlight.

The best thing about Moonlight, is it’s free.

Moonlight

Moonlight uses Nvidia GameStream’s service, which allows you to stream from a PC to a Nvidia Shield, but Moonlight opens up those streams to outside of a LAN and it’s available on multiple platforms too, so we can use Android, Amazon, ChromeOS, Linux, and iOS devices as well. Hell, I might even try a Rasberry Pi if I have some time,

But here is the kicker, being able to remote in and start a game on my PC is meaningless if it’s unplayable. Meaning, less than 30fps or the lag is so bad it severely affects the gameplay.

So how does it play on various clients in different situations? I didn’t see any results from anyone and I knew I wouldn’t know for sure unless I tried myself. So are any of these various streaming devices able to replace my standard “Sit at my PC Desk and play at my computer” setup?

Host Gaming PC Requirements

Per Nvidia’s website.

  • NVIDIA GeForce GTX/RTX 600+ series GPU (GT-series and AMD GPUs aren’t supported by NVIDIA GameStream)
  • NVIDIA GeForce Experience (GFE) 2.1.1 or higher
  • 720p or higher display (or headless display dongle) connected to the GeForce GPU
  • 5 Mbps or higher upload speed (only required for streaming outside your house)

Once Nvidia GeForce Experience and Moonlight is installed, it auto detects Gamestream PCs over LAN. if not, you can easily add by entering the IP address of the streaming PC.

Double clicking on the streaming PC in Moonlight will prompt the application to pair to the PCs, after following instructions on the streaming PC (enter a code to pair), The computer is ready to play.

Testing Methodology

Streaming PC
CPU: AMD Ryzen 7 1700 @ 3.00GHZ
Memory: 32GB DDR 4 2133Mhz
GPU: Nvidia GTX 1070
LAN: 10Gigabit
Storage: 512GB M2 and 1TB SSD
Controller: SteelSeries Stratus Duo Wireless Gaming Controller
(Works with PC, MAC, Android, and iOS)

I am not going to get quantitative in this part of the blog. I’m going purely qualitative here, I know I have enough bandwidth in the network. If I happen to run into a bottleneck somewhere it’s either with the software of Moonlight or my PCs video hardware.

I’m not going to dig into logs and measure network interference and speeds unless I have to. I’m going to just boot up a couple games and play for around 5 minutes a piece, record a couple clips to showcase the feel of it. The fundamental question I’m trying to answer is.

Is this playable? And I don’t mean barely. Can this be an enjoyable experience without noticing the streaming becoming an ordeal?

Does this alter the experience enough to where I would prefer to stream rather than to sit at my desk as I have for years?


Nvidia Shield (LAN over Ethernet)

Last year I bought an Nvidia shield, I’ve always liked the Android boxes. I have mine hooked up via Ethernet because I stream a lot of 4K content. Using the Nvidia Gamestream application on the device, my tested speed is far above the recommended bandwidth of 12Mbps which is surprisingly slim.

It says >30Mbps, but the bottleneck from the Shield to the streaming PC is the Gigabit interface on the Shield. Everything else is cat 7 and 10G, so the actual bandwidth should be much higher.
Sekiro (Max Settings 1080P)

Moonlight runs great, but that’s not necessarily a surprise at this point. Gamestream has been available for a while for the Shield(~2017), but despite that. I find the video quality to be better than I expected. Not as much blocking and compression that I experienced with Kainy. There has been much work done in remote streaming software since 2013. It feels so refreshing to be able to play my PC games on my couch on the other side of my apartment.

All audio is passed through the stream, so I can pop headphones into my controller and it just works. Back in 2013, Kainy would still play audio on my computer speakers. No audio artifacts or slowdown noticed here.

Outer Wilds (Max Settings 1080P)

For most games, Moonlight will automatically optimize the game’s settings to an optimal level, so no, I’m not constantly fiddling around in the settings when i move from one platform to the other. If you didn’t tell me this was a stream, I would have had no idea.

Dell XPS Laptop (WiFi LAN)

CPU: Intel Core i7-8705G @ 3.10GHZ
Memory: 16GB DDR4 2400Mhz
GPU: AMD Radeon RX Vega M GL
LAN: 801.11ac
Storage: 256GB M2

Ironically, this laptop’s CPU is actually faster than the streamer, but the GPU in the streamer smokes the Vega. I can play Quantum break just fine with a controller, but there is the slightest delay. Small enough to where I’m not sure if it’s moonlight or just Quantum break. It doesn’t necessarily ruin the experience though. I feel as though I could complete a whole play through with this setup. Note: Those green lines on the screen are the LED lights in my room.

Quantum Break (Max Settings 1080P)

I primarily got this laptop for the 4k screen and surprisingly, with a little tweaks on my streaming PC, I can stream Hellblade:Senua’s Sacrifice in 4k, which is larger than the monitor resolution of the streaming PC. So I’m able to play this game at a higher settings than if I actually sat at my desk. Albeit, there is little slowdown, which is primarily due to the streaming PC struggling to run Hellblade in 4k. I have fraps running on both computers, and the streaming PC is dropping below 60fps occasionally.

Hellblade: Senua’s Sacrifice (4K, 30FPS, High settings)

But what is the resource drain on the laptop when it’s playing? So just to give you an idea, I changed to play a windowed session in 1080P and found that Moonlight is very lightweight. The CPU underclocks and the GPU isn’t even utilized to decode video so battery life is extended even further. The screen back light will be probably be the biggest power draw on this laptop while gaming.

Notice the underclock (1.73Ghz from 3.1Ghz)on the CPU to save battery. To the CPU this might as well be a single YouTube video. Also, side note, look how muted a screenshot is when it ignores the HDR information that’s in the video.

With Moonlight’s settings at 1080P, 60FPS, and 98Mbps bitrate, Hellblade:Senua’s Sacrifice plays great. This is my first time playing this game and if I sat down with a session already running, I would not be able to notice this wasn’t running on the laptop until I realized that the laptop isn’t screaming trying to stay cool.

Mad Max (Max Settings @ 1080P)

Mad Max was a game I regularly played on the Shield for a few months before getting into this. Plays perfect on the laptop as well, but all of these controller games can hide a little bit of that latency. Let’s try out a mouse and keyboard game. How about Cities Skylines?

Cities Skylines (High Settings @ 1080P)

Plays great, the mouse lag feels like playing a game with a poorly optimized menu. That even feels like it’s being too harsh. It’s playable, but wiggling the mouse back feels a little less than lightweight as normal. Even with that, this can absolutely replace my desktop gaming experience, lets move on to some of the more obscure assets.

Android Galaxy S8+ (LAN over WiFi)

This past E3, everyone was so impressed with The Witcher 3 running gimped on the Nintendo Switch. So let’s see if I can play it on my phone.

The Witcher 3 (High Settings @ 1080P)

Alexander Graham Bell is shitting his pants in heaven.

Seriously though, this is surreal, I’m not going to be able to go to bed at a reasonable time now. High at 1080P is ok, but I still cant read any text, lowering down to 720P makes this game run butter smooth.

To give you a sense of the latency here, here is GTA V playing on Max settings at 720P. The bigger screen is the main monitor for the streaming PC. As moonlight is running, your main desktop displays the game normally. So if not even the screen, you could lower the settings and simply use it as an input device.

GTAV (Max Settings @ 1080P)

Just look at it. GOD DAMN. I just think that’s the coolest thing.

Asus Junk Laptop (LAN over WiFi)

CPU: Intel Celeron Quad Core N3450 @1.1Ghz
Memory: 3.7GB
GPU: N/A
LAN: Wifi
Storage: 32GB SSD (Really)

Last year, I bought the cheapest laptop I could find and it was far worse than I ever imagined. It had a Celeron processor and only 32Gb of hard drive space. It was a completely wasted configuration. Anytime a laptop says onboard memory instead of RAM, you know you’re gonna have a bad time. My first boot and update filled up the hard drive with windows updates. I had to uninstall it and install Ubuntu Mate. Moonlight has a Linux version available, so let’s boot it up and see if I can stream Sekiro to it.

Sekiro (Max Settings 1080P)

Works like a charm. Maybe I didn’t actually need this XPS as much as I thought? Then again who am I kidding, look at that rinkydink screen. For many of us, Moonlight might suddenly become the perfect answer to the “What do I do with this old as hell laptop?” question. I’m suddenly curious what’s the worst computer I can find that can run Moonlight now.

Apple iPad 3 (2012) (Lan over WiFi)

CPU: Apple A9 @ 1Ghz
Memory: 1GB DDR2
GPU: N/A
LAN: Wifi 802.11a/b/g/n
Storage: 32GB

This is probably the most impressive one yet. This iPad is so old and unable to be upgraded that I cannot even download the Netflix app. I have to go through safari. Moonlight is still compatible though, so lets try Sekiro.

Sekiro (Max Settings 1080P)

This is the first instance where I see a slowdown.

Right at the beginning of the video, there’s a sludgey part where the stream slows down and then speeds back up to catch up. The audio became robotic during it, but besides that small slowdown, I haven’t hit any other performance speedbumps besides struggling hitting the fps ceiling in 4K, which is obviously going to be a bottleneck. 4k Gaming is insane.

Seriously, if I type too fast on this iPad it freezes up like a emo kid with crippling anxiety. The fact it’s mostly playable is incredible. Let’s try a mouse game.

If you’re in a situation where you’re toying with other use cases, definitely look into Moonlight. It might be an option you haven’t considered yet. Personally, after trying a couple different platforms and games, I have no problem in saying that streaming my games within my LAN can absolutely replace my setup for some games, in fact, it might become my default style of playing from now on, but streaming outside of my LAN?

Well that’s another blog post for another day. Until then, take care.

Building a Custom Voice Assistant: Part 1

Posted on November 27, 2018

It’s kind of an open secret that I love Amazon Alexa, my friends know it, my girlfriend knows it, hell even my two roommates who I scare the shit out of when I drop in via Alexa to randomly say hello in the kitchen know it. This is all due to the fact that over the past six months, I’ve been slowly offloading small tasks to Alexa. Turning on the lights, setting timers, playing music, etc. I’ve started experimenting with “scenes”, automated actions such as turning lights on an air conditioner on or off. for when I come home or when I leave for work(turning off lights) It’s surprising how much small these small automations have altered how I operate now day to day. Looking forward, I hope to utilize them even more so, but there’s a problem. See, to put it lightly, I’m not really a fan of Jeff Bezos’ piggy bank listening to everything I say. The problem doesn’t lie only with Amazon though. Every single voice assistant operating via the cloud offered by Google, Facebook, and anyone else cannot be trusted to protect your privacy.

So in an effort to minimize my data footprint, I’m gonna build a private voice assistant myself. I’ve seen numerous guides on building your own Amazon Alexa in a custom piece of hardware such a raspberry pi, but those still send queries to Amazon. I want to build a private voice assistant that can replace Amazon Alexa’s functions for me, but can ensure that my privacy is protected at the same time.

To make a long story about decentralization and privacy by design short(er), all of these companies’ business models operate by mining your data and then selling it. Amazon isn’t offering these Echo Dots and Echo Spots at such an insanely cheap discounts for no reason, Amazon intends to milk it for all it’s worth. There’s even a new Echo Auto that’s only $25 that connects to your phone that you can use in the car. Knowing that the Amazon Alexa app requires location permissions on the phone app, I’ll bet my Grandma’s lucky silver dollar that it’ll continuously send that data to Amazon. It’ll send anything it can store, because some have estimated that Amazon will hit $10 billion in Alexa related sales in 2020 alone. Amazon knows exactly what it’s doing and Alexa is going to be the main point of contact between Amazon and their customers in the next decade going forward, so this pull into the internet of things is only going to ramp up going forward.

Every single time I state an action, my words are translated to text, that text is then parsed, categorized, and then stored. I know because I can open my Alexa phone app and see in the history that the other day, my roommates 30 miles away asked the Alexa in the kitchen if she likes handjobs. She didn’t respond, shes always so coy, but anyway, I digress.

A lot of people are guilty of obfuscation when it comes to the cloud by making it seem far more complicated than it is. The cloud is just someone else’s computer, really. It’s also heavily insinuated that these complex voice assistants that are often branded as an “AI” require processing in the cloud, this is argued because it requires too much processing power to computer these commands.

That is not true. It is perfectly possible to process these sort of commands locally on the device.

This is how we’re told Amazon Alexa operates. Requests are processed through Amazon’s API at their servers and then routed through a device’s manufacturer’s network, and then that’s pushed to the device. For example, the RGB led lights in my living room are from a company called Magic Home that requires their own account and sign up process. This opens a can of worms and begs the question, how much info is being shared between Amazon and Magic Home? Is Amazon allowing Magic Home access to a lot more data than they should? Is even my Alexa connected coffee pot also sending private information somewhere?

The way to resolve this is by processing everything locally on the device. Allow my very shitty diagram to illustrate. I have been looking at a number of different solutions and I think I’m going to try out Snips.ai first. Snips seems to be trying to do exactly what I had in mind. Local processing of queries all done in an open source environment so I can guarantee that my information isn’t sent anywhere I don’t want. I also could even unplug my router and ensure it still operates regularly unlike Alexa who has a stroke when you do that.

I know through the Magic Home app that the modules can be manipulated over a local network via an app, the devices themselves broadcast their own tiny wifi network that phones can use to connect to. If for some reason I can’t go that route and I have to include Magic Home’s servers in this process, I will at least have reviewed the messages myself and would implement any sort of compensating control if possible, but I’ll cross that if necessary. If worse comes to worse and I can’t use the small Magic Home LED module with an led strip (found here), I can directly wire the leds manually with a mosfet.

So how much is this gonna cost anyway?

Actually, not that much.

 

Step 1: Hardware

1.) Raspberry Pi 3 Kit with Clear Case and 2.5A Power Supply – $49.99

I found a decent kit that includes most of everything on Amazon here, but damn, we’re already going over our budget. Just tell yourself you’re saving money in the long run by not paying with your data. Amazon is probably Echo Dots at huge losses anyway and there’s no way to financially compete. I went with the pi 3 because I don’t want to have to worry about any performance bottlenecks. I’m not even putting thought into older Raspberry pis. If we can try this config with older pis, we can do that later and perhaps find cheaper kits to use.

 

 

 

2.) 3.5mm Mini Portable Stereo Speaker for iPod

I bought this 3.5mm speaker, but I didn’t realize it required a battery. Don’t buy that one, buy this one. That has a usb that can keep the speaker powered. Also, don’t buy things on Amazon in a flurry.

 

 

 

 

 

3.) TONOR PC Microphone USB Computer Condenser Studio Mic

Looking for something that’s omni directional. I imagine tweaking the microphone setup to find the sweet spot of sensitivity is going to be a chore. I’ve even seen in other guides of people using a microphone array.

This’ll do.

 

 

 

4.) USB Memory Stick

I already have one of these laying around. You should be able to find a microsd online for less than $10. If you don’t have a microSD, you can boot from a usb stick only after you’ve already booted from a microSD. Just get a microSD you cheap bastard..

 

 

 

 

 

Step 2: Software

Take that microusb and install NOOBS on it here….. right after you realize you don’t have a card reader and quickly bought one. Also, now that we’re realizing we’re missing some basic stuff, make sure you have a keyboard and mouse too.

I’m running Debian (found here), in a virtual machine in Oracle VM VirtualBox (found here) on my PC (found in my apartment). If you’re running linux or Mac, you don’t need the virtual machine and can run the commands straight from terminal. After you have the pi booted and running, make sure you enable SSH on it (how to here) and be sure to harden it (guide here) so it doesn’t spontaneously learn mandarin.

Installing Curl, Node, NPM, and Snips

Open up terminal on the virtual machine (or in terminal on Mac or linux) and run the following

curl -sL http://deb.nodesource.com/setup_8.x | sudo bash -

After that, run the following to make sure node.js (at least v7.5,0) and npm are installed.

sudo apt-get install nodejs

Verify installs on both by running node -v and npm -v

Now run the following to install snips and sam.

npm install -g snips-sam

 

Connecting to the pi

Now, in the virtual Debian on my PC I should be able to connect to the pi by running the following command.

sam devices

Ordinarily, sam will list all the devices it detects and you should be able to connect directly to it by running sam connect raspberrypi.local however, in this scenario it wouldn’t detect my pi. Running ifconfig on the pi will display the ip address. Take that ip and run sam connect <ip address of raspberry pi>

sam connect <ip address of pi>
Enter username for the device: pi
Enter password for the device: 
Connected to <ip address of pi>

Login via your pi username and password and viola, you’re connected and logged in on the pi. Anything run on this command line will be executed in the pi. It’s actually downhill from here once you run the following.

sam init

Watch the command line go to work. It’ll take a few minutes so go make a shirley temple in the meantime. Or if you have friends that are easily impressed, you can let them watch and watch them then assume you’re practically Lisbeth Salander. Woah buddy, big league hacker shit here.

 

Configure Snips Hardware

After the install is complete, run the following to get a status on everything.

sam status

You should get something like this

sam status
​
Connected to device raspberrypi.local
​
OS version ................... Raspbian GNU/Linux 9 (stretch)
Installed assistant .......... Not installed
Status ....................... Installed, not running
​
Service status:
​
snips-analytics .............. 0.55.2 (not running)
snips-asr .................... 0.55.2 (not running)
snips-audio-server ........... 0.55.2 (running)
snips-dialogue ............... 0.55.2 (not running)
snips-hotword ................ 0.55.2 (not running)
snips-nlu .................... 0.55.2 (not running)
snips-skill-server ........... 0.55.2 (not running)
snips-tts .................... 0.55.2 (running)

Lets quickly run through the main pieces and ensure everything is working. Runsam test speaker

With you speaker connected, you should hear a voice. I have an HDMI cord running from my pi and I heard the audio through the TV. So the output from the device is working. Let’s move on to the microphone and plug it in.

After plugging that bad boy in, run sam setup audio

This will allow you to select the microphone.

sam setup audio
Starting microphone setup...
What microphone do you use?
[1] Generic USB
...

After it’s selected, run sam test microphone

sam test microphone
Testing microphone
Say something in the microphone, then press Enter...
...

Try recording a quick joke (and press enter) to hear it back and realize just how unfunny you are.

Then run sam sound-feedback on

This adds the “ding’ when you make a command.

 

Install Demo

We’re almost to the end, run sam install demo

This should install and turn on the snips service and load it with a basic test app. The default test app just translates your speech into text via the STT (speech to text) API and then repeats it back with the TTS(Text to Speech) API. Once it’s done installing, Snips is ready to be operated via speech by speaking “Hey Snips, <say phrase to be repeated>” You can probably ascertain it’s not perfect, but its usable to improve upon and Any custom commands and tweaks including my own lighting setup and automations. I’ll document in part 2.

 

For now, you can say, “Hey Snips, the colossus of clout!” and you can marvel that you’ve made a digital Tommy “Repeat” Timmons from The Sandlot.