If they don't have it, they come here.
These CPUs are assembled in China. So
they apparently I guess they have spare
and they buy it from these warehouses
which are on the upper floors of the
building. They just have carton. There's
like probably a dozen tall and about the
same deep.
There's a plumber.
Yep. They're I thought they they're
being full. They're quite well utilized.
They one of them sent me this price
sheet.
We're multiple administrations deep in a
technological cold war over processing
power between the United States and
China. China's cyerspace administration
has labeled some US graphics processing
products as a security risk, seeking
answers about US government back doors
in silicon. Meanwhile, the United States
has imposed heavy restrictions on
exports of graphics processing units or
GPUs being sold to Chinese companies by
American companies. The sale requires
rarely granted licenses for each import
scenario to legally export GPUs above a
certain performance level with the
stated objective being to restrict
progress of private and government
projects, including AI development in
China, while trying to maintain the US's
AI leadership. The United States takes
this so seriously that just this week,
the Department of Justice had two
Chinese nationals arrested in California
for what it alleges is the smuggling of
tens of millions of dollars worth of
GPUs.
Because where there's prohibition,
there's smuggling.
We spoke to everyone about this Nvidia
AI GPU black market in China and the US.
We found middlemen who connect buyers
and sellers.
This is it. Cool. Wow.
Users who can understand the demand and
explain it using the most dystopian
definition of wealth. How many GPUs they
have.
They have much more GPUs than the
university.
Which GPU? How far?
Wow.
And we got them quite cheap. What model?
This is the A1 A100.
Oh, A100.
These are A100. We also found
independent repair shops who simply
doing their jobs salvage valuable
silicon components from dead PCBs of
banned GPUs, innovatively hand modifying
them to be better than and have more
VRAM than Nvidia's own official product
SKUs.
They should make one right now.
Okay.
Hey, cool.
We met multiple people who when asked
the same question gave the same passcode
like Chinese idiomatic expression or chu
Nvidia.
Do you think Nvidia
knows? Do I try to stop it?
And among others, we even spoke with a
US-based Chinese national buying video
guards in the US to strip them and ship
the GPUs to companies in China,
violating US export control law.
I haven't seen someone have a computer
in a car for a long time.
These nervous laughs come from a viewer
of ours while meeting in Arizona with a
GPU smuggler we'll call the plug. The
smuggler spoke limited English, but they
both understood one universal truth.
Oh, the temperature is too hot.
Yeah.
Okay, two universal truths. The second
one is money. In this battery operated
Prius mounted GPU testing rig with fur
mark is the least suspicious thing in
his car with a spare license plate in
the trunk. We'll come back to the plug
later. Black market is normally a phrase
associated with drugs or guns, but
there's a new kind of black market and
it's high-end AI GPUs. And this
particular black market is worth
billions of dollars a year. It's hiding
in plain sight.
This is Hong Kong.
It's our first stop on this journey. By
skyscraper count, Hong Kong would be the
tallest city on Earth. The density is
unbelievable. We spent a few days here
for this story, wandering markets and
meeting sources.
Do you feel like the ban is it targeted
at like military uses where they're
trying to prevent that?
I don't think
from the US government's perspective
there's this distinction between like
Chinese academia versus Chinese military
versus Chinese commercial. I think no I
don't think they they really consider
there's a distinction right
yeah just total yeah origin or band
yeah so these GPUs are almost certainly
moved one at a time
right so it's very hard to get like a
full HGX system right
oh so the 90 normally comes from
Australia and Taiwan
yeah and that's why we're here to learn
what the demand drivers are for these
GPUs how they get into China and more
about the illicit side of the GPU
smuggling business.
The community brought you this movie.
This production is being funded by our
viewers. Rather than putting a third
party ad here, we're supporting
ourselves via the community so that we
can produce more fully independent
movies like this. We launched our own
Kickstarter style fundraiser on our
store linked below. In addition to
physical products like our blind eye
t-shirt, black market GPU sticker backs,
blacklisted t-shirt, and our new GPU
VRAMm glassware, we also have digital
rewards. It's available exclusively to
backers on this page. These backer tiers
are available only for one more week,
featuring bonus videos, wallpaper packs,
and the highest tier even has a Q&A
session with me, a tube of cane pin
thermal paste, and features a canvas
print of a watercolor painting painted
by my mom, actually based on a photo I
took in Hong Kong. We even hit a super
stretch goal where all backers of any
tier will receive digital rewards on a
USB key, including a copy of this very
movie so that you can watch it anywhere,
anytime, without internet, and without
YouTube AdSense ads and preserve it in
case it's taken down. If you prefer a
single item, we have a brand new blind
eye t-shirt featuring traditional
Chinese characters written right to
left, top to bottom, representing the
idiomatic expression we kept hearing
during this investigation.
easier. Yeah,
the shirt features a cyberpunk dystopian
style open eye with a camera as the
iris. Our fundraisers already hit
several stretch goals, and because it's
brought in so much, we're going to
release even more black market backer
exclusive videos that you'll get shortly
for backing this project. All of these
items will ship throughout September and
October as we bring them in. So, place
your orders now to get them when they go
out. The link is below. Now, back to the
program. This was our latest big
adventure, and it began when we booked a
24-hour ticket to Hong Kong. 20 days of
hotels across Hong Kong, Shenzhen,
Janjjo, Hujo, Donguan, Taipei, and more.
At least one of us got detained by at
least one of the governments involved in
this video, but we can't talk about
these indecipherably singular or plural
instances or instance of any
affforementioned detainment. Next, we
talked to a lawyer about the previous
sentence, loaded into a cab that secured
our luggage like this, then investigated
whether or not GPS are actually smuggled
with lobsters into Hong Kong.
Nvidia was right. We've uh we've checked
the lobsters for sensitive electronics
and aren't any in there. So,
good move, Jensen. We found a pile of
GPUs that are now export controlled and
met a professor who really wants to make
sure you know
these are these are legally obtained.
Legally obtained.
Came to a sudden realization that if a
banned GPU is assembled in China,
then what's to stop them from just
taking it off? Actually, very good
question because um what's preventing
these GPUs from
falling off of the line?
Line. Exactly.
Made friends with three different cats
in Hong Kong. What do you think of the
temple guards? Tannon,
they're uh they're very good at their
job and said goodbye to this RGB city.
We then booked a boat to Shilo, China. I
told Tannon what Shilo means.
You know what Shoko means, Tannon?
What does that mean?
Snake's mouth.
Sounds inviting, right?
Yeah. Awesome. Can't wait.
We arrived in Shenzhen. made small talk
with hotel room service.
So, uh, how long have you worked at the
hotel?
Immediately got trapped in the hotel
elevator by my new friend.
Traffic jam.
Almost witnessed a workplace accident.
Got kicked out of a warehouse.
We got kicked out.
Yes, we did. And then we instantly
became fans of using highspeed rail at
hundreds of miles an hour to go deep
into China to meet a guy who really
thinks that desoldering a GPU and
rebaling it is no big deal.
Really? That's the easy part.
Oh, really? Okay. It's a lot easier than
doing this.
We got a ton of information in this
video about GPU smuggling, including
from this guy named Five.
After
that, we went back to Shenzhen for the
third time in two weeks. Met this guy
supported game natures, supported Jesus.
Yeah. Yeah. I'm the he his father.
Went back to Hong Kong, flew to Taiwan,
got some street food, and got on a plane
back to the US. We spoke to a lot of
people for this story to go on record
and learn about the dystopian world of
high-tech GPU smuggling. We spoke to
people ranging from owner operator
trading companies to professors of
economics building data centers.
These are very expensive items,
right? I would imagine you would keep
track of everything, right?
We even tried to talk to the US
Department of Commerce, but they didn't
reply. and every person on the chain
from the Department of State had out of
office autoresponders because we
coincidentally emailed them the same
week of an especially problematic story
that enigmatically involves name
dropping. But let's keep this focused on
the hardware. Some of our sources are
more confused by Nvidia than others
today.
And people seem to laugh every time we
ask about how many Intel GPUs they sell.
But we want to connect as many dots of
the smuggling line as we can today. And
that's why we shot over 12 hours of
interview footage that we just spent
weeks cutting down for time. The point
is finding people who know people. And
each person in our lineup today has led
us to at least one other person in this
video, eventually building the full
pipeline of smuggler to user. Although
this story isn't about drugs, it is
about a different thing that billionaire
executives use to get high from is AI.
AI,
AI, AI,
AI,
AI.
But before we get to any of those
interviews, we need to establish the
basics of this geopolitical mess. The
story is complicated, so we'll start
with defining these key facts. Why these
GPUs are banned, the new 15% license,
which only applies to two GPU models,
who buying and selling is legal or
illegal for, and then the timeline. AI
GPUs have been in even the mainstream
news constantly.
China has spent decades modernizing its
military with the express goal of
catching up to the US. AI enabled drones
are key to achieving that goal.
We still sell billions of dollars a year
of semiconductors to China. We just
cannot let them access the most
sophisticated, cutting edge artificial
intelligence chips.
The export ban did prevent China from
getting the highest end American chips.
But it did not stop China from catching
up in the AI race.
China's AI technology is getting a lot
better. The concerns over Chinese focus
on AI. China's China.
China's China. China.
China.
This story has been a complete mess to
follow. It spanned years and now two US
administrations. We ended up with over
400 pages of research and maps and
finding sources and trying to understand
the laws. Here's why governments care
about AI GPUs. Nvidia functionally holds
a GPU monopoly in our sector of the
industry, which is building computers to
play video games. That now feels
relatively innocent by comparison to AI.
The company leveraged decades of gaming
domination to build a foundation for
what it now focuses on, which is making
the most powerful GPUs for AI in the
world. Reports warn of Nvidia product
use and nuclear weapons research, facial
recognition technology allegedly used in
Russia to suppress descent, and growing
concerns of AI facial recognition use in
the US for similar deployments,
alongside reports of use in
international spying and in drone
warfare. Nvidia finds itself in the
middle of all of this. Even though
Nvidia disputes selling to some of these
entities, for example, it says that it
doesn't sell GPUs to Russia, the
products still find their way there. So,
Nvidia is making money one way or the
other. Someone is buying it, maybe from
someone else who bought it from someone
else. And much like we're talking about
today, it might be transacted through
smuggling. But Nvidia does end up
selling the device ultimately to
somebody. Regardless of who they sell
to, Nvidia plays a big part in this
worldwide obsession with AI. AI, AI, AI,
AI, AI, AI, AI, AI, AI, AI, AI, AI, AI
will also create AI using AI to create
AI.
And we think it's playing all sides.
We'll talk about that more at the end,
though. And besides, when there's a gold
rush, it's better to sell the pickaxe
than swing it.
There's so many companies that would
like to build. They're sitting on gold
mines. Gold mine. Gold mine. It's a gold
mine. Gold mine. gold mine. It gives me
tremendous joy if you just buy something
from Nvidia. We're going to be rich.
Although the US doesn't talk too much
about its own use of AI, it spends a lot
of time talking about China's.
These semiconductors are unbelievably
powerful and we can't let them get into
the wrong hands.
We want the most other countries using
our technology because again, it's a
zero- sum game. The concern of course in
Washington and particularly amongst the
defense department and the Pentagon is
that these chips and this hardware and
software ends up with the people's
liberation army, the Chinese military
and will help them accelerate in terms
of innovations on that front.
The restrictions are really on advanced
semiconductors that could have um
potential military applications and the
goal is really to sort of choke uh
China's AI sector and and its broader AI
ambitions. The US restricts Nvidia's
GPUs through export control rules that
ban the sale of certain GPUs into China.
That means the sale of GPUs by American
companies or companies which want to
transact business in America to
companies that are in China or the
Chinese government itself. Some examples
of export control GPUs include gaming
GPUs like the RTX5090 and the RTX490
which are useful in AI applications
mostly for their high VRAM capacity. It
also includes specific data center and
AI GPUs like the A100, H100, H200 and
B100 with the letter being the signifier
for the architecture as well as other
GPUs shown on the screen. Now this list
is in flux. There are some new and
incoming exceptions for the Nvidia H20
specifically, which has faced
Schrodinger's GPU ban depending on
whether CEO Jensen Juan had a $1 million
dinner at Mara Lago with Donald Trump on
a given week. A timeline would help, but
we have 9 years to recap. So, we'll do
it quickly. Even at the end of the Obama
administration in 2016, the US
government was just starting to talk
about AI in relation to national
security in this wired interview.
developing
international norms, rules, protocols,
verification mechanisms around cyber
security generally and AI in particular
uh is in its infancy. You got a lot of
non-state actors who are the biggest
players.
Part of the problem is is that
identifying who's doing what is much
more difficult. Yeah. If you're building
a bunch of ICBMs,
we see them. Uh if somebody's sitting at
a keyboard,
we don't. And so uh you know, we've
begun this conversation. uh a lot of the
conversation right now is not at the
level of
you know dealing with real sophisticated
AI but has more to do with essentially
states establishing norms about how they
use their cyber capability. Who are you
more afraid of big brother in the state
or the guy who's trying to empty out
your bank account? Uh, part of the
reason that's so difficult is that if
we're going to police
this wild west, whether it's the
internet or
AI or any of these other areas, then by
definition, the government's got to have
capabilities. If it's got capabilities,
then they're subject to abuse. And uh,
at at a time when there's been a lot of
mistrust built up about government, that
makes it difficult.
Those were the early days. The first
Trump administration started a
commission advising Congress on
maintaining AI leadership, including
simply banning the sale of some advanced
semiconductor equipment and chips to
China. Several years of back and forth,
a pandemic and election and chat GPTs
launched later and the Biden
administration took major action in 2022
by restricting exports to China, Hong
Kong, and Macau. Nvidia's Ampier
architecture A100 and newer code name
Hopper H100 GPUs and systems were all
restricted. Nvidia shed tears for $400
million worth of lost sales as a result.
And it was especially sad when many of
its export compliant alternatives to
these also got banned, like its newly
created A800, H800, and L40S in addition
to Nvidia's RTX 4090 gaming card. Nvidia
said it didn't expect quote near-term
meaningful impact end quote on its
financials. Nvidia responds by designing
a workaround to the previously worked
around workaround leading to the H20.
Then the government adds new rules for
high memory bandwidth cards and the
Biden administration tried to come up
with an AI chip diffusion rule that
would limit the quantity of GPUs being
sold into different countries rather
than only by processing power metrics
because the government really didn't
know how the [\h__\h] to measure these
things and Nvidia knowing more about
GPUs could tweak any dial it wanted to
just barely be compliant. Then Deep Seek
came out, everyone panicked, stocks
plummeted, and the government
scrutinized the role of Nvidia's GPUs in
it. In February 2025, fiscal year 2025
results are posted and Nvidia's
Singapore revenue rockets to 18% of
total revenue based on customer billing
location despite shipments to Singapore
being claimed to be less than 2% of
fiscal year 2025 revenue. Causing people
to say, "Wait a minute." Unrelated,
several GPU smugglers are arrested in
Singapore one day after the fiscal year
report was posted, causing people to
say, "Ah, that makes more sense." AMD
spawns out of nowhere to say it wrote
down $800 million of inventory due to
export controls. Nvidia 1 ups it with a
write-off of $5.5 billion dollars. May
2025, Trump implements wide sweeping
tariffs and rescends the Biden chip
diffusion rule that would have limited
how many AI GPUs Jensen could sell to
other countries. Jensen says
it's just an incredible vision. I think
this is going to be a transformative
idea for the next century for us. These
two initiatives are completely visionary
and it's going to be transformative for
America.
The H20 GPU that was created to comply
with the rules was still okay. Then
Jensen had dinner with Trump at Mara
Lago for a million dollars. Then the H20
got banned. Must have chosen a really
bad restaurant. In July, Jensen Juan
meets with Trump and is permitted to
sell H20s again. Presumably choosing a
better restaurant this time. Juan goes
to China. China says Jensen's GPUs have
tracking devices back doors. Nvidia
says, "Nuh-uh." Jensen goes to
Washington. Trump tells Jensen to
Jensen. Will you stand up? What a job.
He loves him. He loves
Nvidia.
He loves Lisa Sue. And more importantly,
he hates Intel CEO. Intel CEO goes to
Washington. Trump likes him now. Tim
Cook materializes from the Infernal
Plane to give Trump a 24 karat gold
gift. and gift is in scare quotes
because that's not what that's called.
Then we get to this past week when Trump
asked Nvidia and AMD to pay 20% to the
US government for sales specifically of
the H20 and AMD Instinct Mi308 sales,
not all GPUs, as some erroneously
reported previously. Cousins Jensen Juan
and Lisa Sue negotiate Trump down to 15%
and now they're allowed to sell two
specific cards that were originally
created to comply with the laws before
they changed. And somehow everyone walks
away a winner. Except now China doesn't
want them anyway. That about sums it up.
Except one last thing that happened as
we were filming this. The Department of
Commerce doesn't yet know the legality
of the deal with Tom's Hardware
highlighting legal expert arguments over
Article 1, Section 9 of the
Constitution. That's not our area, so
we'll move along from it, but just know
that this is being publicly discussed
still. If you want the full details with
all of the in between, we did collect
it. There will be a massive timeline
article on the website. We'll link that
below when it's up within a couple days.
There was also recent news about a 15%
revenue share between Nvidia and AMD
with the United States government for
sale of some AI GPUs.
This is an old chip that China already
has
and I deal with Jensen who is a great
guy and Nvidia. Uh the chip that we're
talking about, the H20, it's uh it's an
old chip. uh China already has it in a
different form, different name, but they
have it. Uh or they have a combination
of two will make up for it and even then
some. But the H20 is obsolete. You know,
it's one of those things. But it still
has a market. So I said, "Listen, I want
20% if I'm going to prove this for you,
for the country, for our country, for
the US. I don't want it myself. You
know, every time I say like uh like 747,
I want I want Yeah. for the Air Force.
So I just wanted so when I say I want 20
I want for the country I only care about
the country I don't care about myself
and he said would you make it 15 so we
negotiate a little deal so he's selling
a essentially old chip that Huawei has a
similar chip a chip that does the same
thing and I said good if I'm going to
give it to you because they have a uh
you know they have a stopper what we
call a stopper are not allowed to do it.
A restricted is really known as a
restrictive covenant.
If you were to only read the headlines,
you might think this applies to all GPUs
and that the ban is over. And then you
might think that a market in China or in
the US's eyes an illegal market in China
would cease to be so illegal. That's not
the case.
I release them only from the the H20.
This new 15% revenue share would, if
it's legal, and they're not sure yet,
allow Nvidia to specifically sell the
Nvidia H20 GPU to approved Chinese
entities. they likely can't be on the
entity list. It would allow AMD to sell
specifically the Instinct MI308 GPU to
approve Chinese entities. The proposed
license would not affect any other GPU
that currently does not have a license.
The government hasn't made clear yet if
Nvidia's partners would also be
permitted to make these sales. That
means that other band GPUs, including
the RTX 4090, RTX 590, H100, B100, B200,
and so forth, remain banned. The H20's
extremely high 96 GB memory capacity
would enable large models to fit in
memory and actually run, especially with
multiple GPUs in a single rack, even if
it's slower. That means companies can
achieve performance targets by stacking
GPUs, which are lower clock and core
count, but higher capacity, and doing
Nvidia CEO Jensen Juan's favorite thing.
We were trying to solve scale up,
scaling up, scale up, scale out, scaling
up, scale it up, scale it out, scale up,
scaled up, scale up, and scaled it out,
scaled it up. Scaling
in that way.
As for the newer Blackwell architecture
GPUs,
now Jensen also has Jensen's a very
brilliant guy, and Jensen also has a new
chip, the Blackwell. Do you know what
the Blackwell is? The Blackwell is super
duper advanced. Let's not give Nvidia
any ideas for new GPU names.
I wouldn't make a deal with that.
Although, uh it's possible I'd make a
deal uh a somewhat
uh enhanced in a negative way Blackwell.
In other words, take 30% to 50% off of
it. But that's the latest and the
greatest in the world. Nobody has it.
They won't have it for five years on the
Blackwell.
I think he's coming to see me again
about that. But that will be a uh
unenhanced version of the big one.
For now, these super duper GPUs are not
licensed for sale. In short, the H20 and
MI308 were compliant with the US
government's original rules. Then the
rules changed during design and
production. They were banned. Then AMD
and Nvidia collectively declared over $6
billion in financial impact as a result.
Then the US government said, "Wait a
minute. We can help with that if you cut
us in." It reminds us of our tariffs
documentary. And with this, if I see a
tariff of 20 or 40% one week, 50 to 60 a
day later, then I see it 120%. Then I
see, well, electronics are maybe exempt
and in Bloomberg and then I see on Truth
Social, no, that's not actually
happening. I have no idea what that
means. So, I can't make decisions
because it's not about it being a high
tariff. It's about not having a clue
what's going on.
The H20 most certainly is not
obsolete,
though. It's still very desirable in
China, and with a lot of them, they
become particularly potent. Setting a
threshold for banned hardware should be
objective since it can be tested. This
is a graph from the Department of
Commerce that visualizes the original
threshold at which a computing product
became automatically banned for export
to parts of the Middle East and China
without a granted license. There have
been some changes since, but back when
this was made, the total processing
performance score on the Yaxis was used
to determine cards in need of a license.
The government needed a metric to
calculate against, so it created its
own. Accelerators and video cards have a
lot of metrics in their spec sheets,
including memory capacity, which is
critical and as simple as a pass fail
for certain AI and training use cases.
They have hard numbers for memory
bandwidth, GPU clock speed, GPUs or CU
count, TPCs, tensor cores, ROSS that are
sometimes randomly missing on Nvidia
devices. Teraflops, TOPS, pedlops,
gigaflops, gigabits, power, and more. So
then banning a product could probably be
based on some sort of benchmark rather
than a random metric from a spec sheet.
But the US government, illustrating what
an absolute cluster [\h__\h] this situation
was and now remains, decided to instead
multiply one random metric from a
spreadsheet against the bit length of
the operation being executed. Flops or
floating point operations per second and
tops or terra operations per second are
calculated by the company making the
spec sheet and aren't a great measure of
actual performance. These numbers are
based on both marketing and whether
we're talking about FP8 half precision
floating point 16 single precision
floating point 32 double precision FP64
integer performance or tensor
performance or otherwise. So the
government says quote the rate of MAC
tops is to be calculated at its maximum
value theoretically possible end quote
and quote the rate of MAC tops is
assumed to be the highest value the
manufacturer claims in brochures for the
integrated circuit end quote. So it's
not based on a bunch of real world
benchmarks of applications or something.
The government also references Mactops
as the theoretical peak of terra ops in
multiply accumulate computations. The
Biden administration used this total
processing performance or TPP score in
2023 with the government later adding a
performance density metric dividing the
TPP by the die area in square
millimeters. In other words, the
government didn't want Nvidia to be able
to sell more of a lower performance GPU
to make up for the loss of high
performing parts with multiGPU
solutions. There are a lot of reasons
this doesn't capture the full picture
like sparsity, APIs, different methods
to calculate flops and different
performance for different applications.
But the government needed a way to
define a threshold. So this is what they
made. The limit was a TPP score of 4,800
exceeded by even the RTX 4090 when
calculated using tensor performance. Now
if this doesn't mean anything to you,
that's okay because it probably doesn't
mean anything to people signing the laws
either. Or maybe that's not okay, but
you get the idea. Something like a
higher memory capacity, lower flops
performance GPU, or even series of GPUs
like 30, 60, 12 GB cards might be able
to get the work done more effectively if
it only needs memory. Memory wasn't
factored in to the TPP calculation. That
was the point of the Nvidia H20. But
then the absurdity of the situation
expanded because the government
introduced an opaque memory bandwidth
requirement. per the register. Quote,
"Unlike with previous export controls,
the Commerce Department's Bureau of
Industry and Security hasn't issued
specific guidance on how much IO or
memory bandwidth is too much." End
quote. But they did decide that there
would be a limit. So to recap, it seems
like across two administrations, the
United States is creating formulas based
upon numerical calculations, and then
when Nvidia and AMD tweak the numbers to
fit within that box, it's recconing
those rules in a guess and check
process. We spoke to a lot of people in
this piece and it's multilingual which
made it complicated. In order of
appearance today, the discussions come
from Dr. Vinci Chow, senior lecturer in
the department of economics at the
Chinese University of Hong Kong,
responsible for building his
department's machine learning servers
and sourcing the GPUs from middlemen
suppliers. Dr. Zu Ha Fu, a research
assistant professor at the Chinese
University of Hong Kong and an
affiliated lecturer at the University of
Cambridge. Zuhao specializes in both
computer science and linguistics.
various retail workers at computer
markets in Hong Kong to get a ground
level understanding. Joe, the creative
director of product at video card
manufacturer Yeston, who worked with us
for a separate piece, not this one, but
he showed us his factories and
suppliers. So, we've included clips
here, as we relied on those tours for an
understanding of how GPs are brought
into China for assembly on PCBs. An
anonymous seller who goes by the
pseudonym of Silver based in Shenzhen,
Paan, and managing a warehouse in Hong
Kong that receives and processes
smuggled GPUs. Vincent, a resourceful
fence who lives next to the warehousing
and markets filled with GPUs and
accelerators. He buys and sells these
devices in Shenzhen Hua and his job is
to know people. In addition to Vincent's
cousin, the fixers fixer. If someone
needs a component to fix a broken video
card or to build a new one, they go to
people like him to get integrated
circuits. Mr. 5, a Billy Billy hardware
reviewer with a specialized focus on
thermal solutions. Like us, Mr. 5 has
had run-ins with Nvidia that have ended
in a soured relationship over disputes
regarding independent reviews and
editorial independence. Junga, a
renowned Billy Billy uploader, basically
a YouTuber in China who runs a video
card repair shop in Jano. In addition to
repairs, he regularly gets large orders
from customers asking him to build them
custom unofficial higher VRAM capacity
Nvidia GPUs for large language model
tasks. Companies in Singapore and Taiwan
who act as intermediaries between
Nvidia, Nvidia's partners, and companies
in China. The Singaporean and Taiwanese
companies are able to bring banned GPUs
and servers into their own warehouses
and resell them to Chinese companies,
skirting export controls. We're unable
to disclose their identities because
there would be punishment from multiple
governments and Nvidia. The Plug, a
US-based Chinese citizen who drives
around the country buying hardware from
American end users and resell it to
companies in China and Hong Kong. And
finally, a special thanks to our
translator, Raymond Woo of the Black
Soda Agency in Taiwan. Here's the
pipeline. If a GPU doesn't fall off the
back of a truck in China after it was
assembled, or if it isn't a QC defect
that disappears from the scrap pile, it
may instead have been moved by ants to
get to China. May Banana is a linear
pipeline. Each ant in the fire line
serves a specific role. It'll help to
name those roles to keep everything
straight. Here's what we came up with.
The source has access to GPUs. This
could be as innocent as you unknowingly
selling your card on Facebook
Marketplace to the plug. The plug is
responsible for acquiring from the
original source and reselling the
hardware to the chinabased distributors.
In between there's a mule. Sometimes
this is the plug himself doing a trip
home. We learned that overseas students
also regularly return with what are
feasibly defended as personal GPUs that
they bought at Best Buy and then may get
resold for markup and profit. In either
case, the mule gets the GPU into the
country either by shipping it without
interception or by hand carrying it.
Next is the middleman, receiving the
GPUs and often interfacing with or
managing the warehouses that store the
cards. The middleman buys from multiple
plugs, including factories that get rid
of rejects with fixable or unimportant
QC defects, then sells those devices to
more localized distributors. That's when
we get to the fence, who buys and sells
GPUs between middlemen and warehouses to
end users in China. Then we have who
we're calling the fixer, except this
time it's literal. The fixer is an
optional step that may involve soldering
and modifying a GPU to improve it beyond
its original specification and make it
more marketable for domestic AI uses.
They might also just fix QC defects from
factories. Finally, we have the user.
This is self-explanatory. The user is
the demand driver, and often times large
enterprises want dozens or hundreds of
GPUs or more, while smaller users like
the university may just want individual
units or small batches. With everyone's
role named, let's continue. And now we
start our journey. First stop, Hong
Kong.
Upon arriving in Hong Kong after
checking out the local wildlife, one of
our GPU dealer informants slipped us a
price sheet for a mix of GPUs that are
both export controlled and not. All
right, so check this out. This is a GPU
price sheet where we uh just got to Hong
Kong and I've already been able to find
a supplier who works as a middleman
between smugglers. And you just pick the
GPU you want. So you see 5090, 4090,
4090, 48 gig, 5090. Down here there's
some A6000, A100, A180 gig. That's
pretty valuable. So that's 75,000
rendering B. And so I found uh a
supplier where basically the the person
who provided this is local and they work
with the smugglers. So they sit between
the smuggler and the customer to supply
and source GPUs. Problem is I don't know
what the going rate is for any of this.
And so what we're going to do is take
this price sheet to our local fixer Dr.
Vinci Chow at the Chinese University of
Hong Kong and see what he thinks about
the prices here and if this process is
normal. So for that, we hopped in a
classic Hong Kong cab and headed across
Victoria Harbor over to the Chinese
University of Hong Kong.
Oh, okay. S. Okay.
Thank you, sir.
Byebye. Bye-bye.
We found Dr. Vinci Chow through a
Reuters story from a couple years ago.
Vinci works in the economics department
at the university and he's responsible
for having built many of the servers and
machine learning systems that are in use
daily.
Here are some professors. Oh yeah,
Vinci.
He has a whole blog detailing his
process, including the difficulty of
sourcing components and the ease with
fixing problems when so close to
Shenzhen, such as having custom PCIe
riser cables built and basically done
the same day. Dr. Vinci Chow is the
right person to start us off with this
story and educate us on where to go
next. We're here to learn the user's
perspective because before there could
be any kind of market for it, there has
to be demand. And that is what the
university and organizations like it
generate.
Hi. Whoa.
How you doing?
Hey, nice to meet you.
Good to meet you. This is Tannon.
Hi.
Nice to meet you, sir.
Right. So, uh we have some rocking. Uh
we can set up shop here. It's a
construction site here right now.
Yeah.
So, we are renovating.
An interesting anecdote is that Dr.
Vinci Chow actually went to school with
someone important at NVIDIA today, the
vice president of applied deep learning
research.
I was actually in Brooklyn with same
time as Brian Karo.
Oh, cool. We're 14 buddies.
That's cool. Yeah, I know him.
Wow.
We won't be able to do much talking in
here because the noise
these are these are legally obtained.
These are legally obained.
We got it. Actually very interesting
because we got these I think uh right
before
the van.
Uh no right before chat uh Chad GBD was
released.
Okay.
Around several years ago.
Yes. And we got them quite cheap. One of
them was actually modified from uh SXM
before everything GPU becomes such a
scarce resource.
Yeah,
there was a period of time I think right
after crypto like crash.
Yeah, that GPU was actually quite cheap.
Even A100 was actually quite cheap.
We were able to get these for like
less than $10,000,
right? And then Chinese chat came out
and everything like that crazy. Yeah.
Even now like if you go on go on like
AliExpress like you still see a lot of
2080 Tai like 2080 Ti being modified. So
for teaching we use a 3060
but because without modification in the
in a gener in generation it has the
highest memory capacity.
Right.
Right. Uh but it's all gone like clearly
there's demand. It probably makes sense
for us to get the mics on. Because of
the way we shot this, you get to see
something you don't normally get to see
in our stories, which is all of the
conversation leading up to formally
filming. Sometimes there's more
interesting information when we're just
spitballing.
I guess even universities have become
more secretive,
right? In the past, you probably
universities have been very open, right?
In terms of what like what we're doing,
what we what we are capable of. These
days, I think there are second thoughts.
M
some yeah some some researchers might
not want to be so being so open about
yeah the collateral damage is in
right and uh
topic
like uh on the other hand like it's
interesting because
you know it's the it's export ban right
so it's illegal to export
right
to China
and that'll be one of my questions for
you
without without permit right but then
it's completely legal on outside
right there's nothing that says that you
cannot buy high-end GPU,
right?
So from our pers from our end like uh as
long as we follow all relevant
procedures, right? There's absolutely
nothing illegal about for us over here
to buy these GPUs, right? So that makes
it for a very interesting like um
environment.
Yeah. Yeah like like like so you has
because if you look at the like
university
like uh documents right I I I think you
will be able to find like obviously
these universities have been purchasing
GPS right it's completely legal over on
our side right yet this is
clearly not supposed to happen
right from the perspective of the US
government
right
so um for un so in I For universities,
uh it once again it depends a lot on how
committed
like a particular researcher or a
particular unit is when it when they
when it comes to approving GPUs, right?
I mean
if you pay enough
supply is there,
right? Maybe not if you want to build
like like a super cluster, right? Like
for research, most researchers are
talking about one or two GPUs, right?
And if the if they can they have the
funding then it's possible to obtain
that. So it just just everything is more
expensive.
That makes sense.
I'm in the department of economics which
is kind of interesting I guess for
someone in the department of economics
to talk about GPUs but uh yeah we
started uh building our own cluster
during the coit. Right.
Well not just we but you specifically
well me. All right. So I I'm responsible
for that. So it's we because the
department has to fund it. Right.
Right. So, I am the one who plans these
things out, but the department funds it.
I started building my own desktop
computers back when I was in high
school.
Cool.
Right. Uh Hong Kong is a small place,
right? Uh you know, I guess we'll visit
that later on, but in Hong Kong, you can
actually phys physically visit computer
malls where you can just pick whatever
parts you want. And then uh during COT,
then we had this idea that uh we have we
probably have to try to provide remote
access to computing power. The first
GPUs we have was 1080, okay, that we
obtained from the computer mall. $500
each. Back then I was thinking like
that's so expensive. $500 for a GPU. If
you were to ask a local like one of the
local supplier how much u let's say H200
is going to cost
it's going to be about um
30k
US.
30k US. There's a big premium for data
center GPUs, right? So even if you
obtain like a aging A1 A100 that's still
going to be almost
20k.
Is anything from Blackwell over here
yet?
No.
Not yet.
Not yet. Not yet. Not yet.
It's illegal for a company like Nvidia
to sell them into China.
Right.
But not for anybody here to buy them.
Yes.
I messaged some sellers on Alibaba,
right?
And uh
they one of them sent me this price
sheet. Woo.
Tell me what what do you make of this?
Do you have any H200s in your servers?
Well, we want some.
Okay.
We want some.
I can connect you to the supplier.
We want some. Right. We want like that.
We Yeah.
After tax, they want 213,000. What is
that in US? Let's see. So, that is
$29,700
US.
Uh,
yeah, that's about I think they're right
for
I don't know what MSRP is or if they
even have one. Uh I think but first of
all it's not like there are a lot of
companies even in the states they would
just like put out a MSRP and they just
buy it online or something like that
right but I uh I think for the few
companies that actually like list a
price
I think it's actually the premium is to
as I said it's surprisingly low
right
right it's almost as much it's just
about the same as the MSRP as far as I
can tell
do you know h how these companies like
the suppliers get the GPU use any idea
cuz I've I've read a lot, right? Like
I've read that um
I I came across a Chinese phrase in one
of the um by do searches I did
and it I think it was like my banana or
something
and I the idea if I understood it
correctly was that a bunch of people
kind of move one GPU.
So these GPUs are almost certainly moved
one at a time,
right? So, it's very hard to get like a
full XGX system, right? These days, most
of
recommend getting the PCIe versions of
the GPUs
because it's so difficult
to smuggle in a full HDX system,
right?
Okay. Yeah.
Um now if you want to actually exist
them for example the companies that have
ways of obtaining them would recommend
that uh they be in stored in a different
country.
Okay.
Right. So basically you can like uh
acquire HX and then have it installed in
like Southeast Asia.
Okay. So meaning
it would actually physically located
somewhere else and then you remote into
it.
Right.
That's what they would recommend if you
want that. If you actually want it on
premise,
right? If you want it here,
then almost every one of them would
recommend that you just get the PCIe
version.
Okay.
So, that probably
just because it's easier.
It's easier
when you put in an order to buy
something.
Um, is it already guaranteed here at
that point
or do they have to get it?
Uh, depends.
Okay.
Some companies have limited stock. We're
talking about probably maybe a dozen.
Right. So, but uh if you want more then
you would have you Yeah. It would
probably have to be like ordered and
like a deposit or something.
Yeah. Shipped in from somewhere else.
Any idea what happens if they get
caught? I mean to your money, you know,
like
Well, we don't pay.
Okay.
Until Okay. Actually, we have to pay
deposit, right? But then, uh I presume
if they cannot deliver
you maybe get it back. we will be able
to get it back locally since uh I mean
we're not doing anything legal so we can
just go to the court right
and suit them and
do the local governments even care I
mean like cuz
I guess that was one of the things that
was unclear to me where if it leaves the
US like let's just say
and and I as an example if Nvidia
decides you know what we're just going
to ship it straight from Santa Clara
California to Shenzhen or something you
know that would be a problem right but
on the rece receiving side, do they
care? You know, does like Hongong
on the rece I would say the receipt.
Well,
I would say no that we don't Well, we
probably don't care, right? Um I don't
think the government is going to
obtain these GPUs now that there's an
export ban. I don't think they're going
to do that. In Hong Kong, we have like
basically the government essentially
funded two H800 superport,
but they are both legally obtained like
right before the the ban. So,
and since then um I there hasn't been
any news that they're going to expand
uh their their capacity. So, I would say
the government probably not going to do
it them itself.
I guess the only thing they care about
is if someone brings it in, they pay the
import taxes. Well, but there's no
import tax in Hong Kong. We are free
trade port.
Okay.
You can pretty much bring whatever you
want like like except like I mean yeah
things normal things like firearms like
you can't do that, right? But otherwise
Yeah, we are free trade port.
Okay, that makes a lot more sense then
cuz I think I read that Macau is the
same maybe free free trade port.
Free trade port.
Yeah. Okay. Interesting.
But I'm pretty sure these GPUs don't go
straight from Hong straight from the US
to Hong Kong,
right? They would go through
they probably go through some
San like
right some like third party count like
country somewhere you can actually
legally export those GPUs to
Singapore comes up a lot.
Yeah, Singapore comes up. Malaysia is
another place that comes up,
right?
I saw have you heard of uh I saw a story
from the economist actually
uh where they said now there's like they
they referred to it as I think data
smuggling, right? So they're talking
about bringing data like from
countries with the export ban to
Singapore, right, to have it processed
locally.
So I think it's it's similar to what the
some of these companies have off have
offered us to for consideration is that
uh given how hard it is to bring in a
large system,
it's actually easier either to like
either to fund a system in Malaysia or
Singapore
or you just rent capacity there,
right? and then just remote access,
right? I mean, I guess if the reason why
you actually have to like
take physical hard drives back and forth
is due to the size of the data. It's
going to be hard to transfer
transfer data
remotely. But uh but otherwise the idea
is the same right given that you cannot
bring the system in just have it
somewhere you can access.
How you doing Steve?
Hi I'm Steve.
I'm good to meet you here.
Yeah, good to meet you.
What department are you in? Uh I'm
currently working in the linguistic
department.
Okay.
So my focus is still doing some
programming and running on GPU pre my
background is computer science. So I
need a lot of GPUs,
right?
So you've already got the computer
science background but then
Yeah. Yeah. I got a computer science
background in in this university. Yeah.
Do you have your own server cluster?
Like is it separate from the econ? Yeah.
Yeah. I think they have the best one.
Okay.
This is the best one in this university.
So, who who built the uh the one for the
arts department?
Uh actually we use the university's GPU.
So, university have a central GPU and
everyone can use that if you buy some
GPU and they can help you to manage that
GPU.
Okay. Yeah. So that that's how we are
using but I believe they they have much
more GPUs than the university GPU
richer.
GPU rich.
Yeah.
What um what are the things you guys use
them for normally?
Uh just training and inference uh latent
models. Okay.
Uh like for me my research is about uh
latent model.
Yeah. So if you want to train a model,
teach the N model to gain more
knowledge, you need to fine-tune. And
before that, maybe few years ago, we
just tune from scratch. But now it's not
that popular, but we still need GPU to
fine-tune models. Okay. So this is why
we need that.
What um what are the common uses for the
models themselves? Like do students use
them for research or what's the
Yeah. Um if you are in the university so
meaning for research
okay
so you know currently because a large
bank model can do a lot of things a lot
of research department will focus on
using that
so a lot of research will be focused on
how can last model help them for example
like cleaning the data
and doing some bical inference or just
uh imitate human to do some research
okay something like this so it's will do
you ever end up working with each other?
No, we haven't. Well, we our two
faculties are in talk of co-op
collaboration
but uh as you said like a lot of demand
really coming from uh researchers
that are using these models
for things like uh processing large
amount of text data, right? So like in
the past what we would do is that uh
these re uh researchers would hire
junior research assistants to go through
text data manually and it's a very
expensive and slow process right these
days all you have to do is to feed the
text data to OM
and that really drives an explosive
growth in demand for these RMS and
because open neither neither open AI
noric is interested in serving Hong
Kongm M
which is part of China
right so
that means that there's a lot of demand
for onrem solution
what um so in linguistics do you use it
for just as an I I don't know a lot
about linguistics research but uh well
the Dwan laer is a she's a linguistics
researcher and um
so I know one of the things that they do
in the university of Columbia in New
York is they'll scan old documents from
like the medieval ages from a thousand
years ago, right? Cool.
And they try to convert it into modern
English.
So what
what kinds of research I guess to give
me a better idea.
First one
h if you want to do research on
linguistic you will be very interested
in what the language model what how the
language model are thinking right. Okay.
So we try to explain what happened
inside the language model and another uh
research direction maybe is just uh use
that language model to processing the
data. Uh for example you want to know
how the children learn the language.
Okay
children learn uh bilingual or many
languages. So how how they do it. So you
can use the length model to imitate the
learning procedure. So this is one of
the
uh possible direction. So there will be
a lot of things to do.
Yeah.
What um what kinds of GPUs do you guys
use?
Uh currently for our group we have A40 I
guess. Yeah. With with 40 GB memory
maybe I forgot.
Sometimes we have we have a
6000
and uh then some uh gaming GPUs RTX
series.
Right.
Yeah. The gaming GPU is much faster
actually.
Is it really? Okay. Do you have any like
5090s yet or 4090s?
I I have used the 4090 but not 5090. I
think memory is the most important part
for researchers because if it's running
slow then you can wait for one day or
two days. It doesn't matter. But if the
memory is small then you cannot do
anything. You cannot put your model
inside the GPU.
It just doesn't run at all.
Yeah. So, so I need larger memory.
So the speed is okay. I think currently
is fine.
Do you guys ever get bottlenecked by uh
the availability of the systems?
Well, it does happen, right? So like as
much as we have as much GPUs as we have,
we talking well we have like 30
something like 30 top of the line GPUs.
Yeah.
If we want to load say uh DeepC1,
right? That takes
probably 60,
right? That's half our capacity,
right? And if so if someone like one of
our teachers or what would say I need
deepseeek R1 for two months
right
then it would take a bit of planning
because we have we would have to see if
we can balance the need of other users.
It's not going to be enough if you want
state-of-the-art model given how big
these state-of-the-art models are. I
think all researchers would love
to scale up what we do but uh the
reality is
uh it might not be possible to get the
hardware to support
what we want to do
if if you had uh and I know you don't
but if you had unlimited budget right
would you be able to get as many GPUs as
you want or is there still a supply
constraint
you know in terms of buying
let's just say somehow we obtain like I
don't millions of dollars of funding,
right?
Right.
Even if the university is providing
that, right? There's still the problem
of how of the export ban,
right? Um
before the export ban as um it can be
done, right? Uh the there's another
university, a local university, HUSD,
they obtained a full H800 super port.
Yeah. at a considerable cost, but the
university was able to fund it and they
managed to got managed to just get it
right before the ban,
right? So that's like kind of a once in
a lifetime
opportunity
opportunity right now, right? Yeah. From
today's perspective, right? And they did
it and they benefit from it. Their
researchers get
a full 400 GPU cluster
to work with, right? It's universally
wide, centrally funded, right? in CK.
Yeah, it's not some it's just not
something we can do right now, right?
Yeah, I think in CK it's managed by each
group,
right?
I believe the computer department also
have a very big one.
Yes. So, so here you know that's but
that's our university. It's a bit
decentralized. So, yeah. And uh part of
the problem as I've mentioned is the
fact that with the export ban uh
departments are not necessarily willing
to publicly
list their cap uh their computing
capabil capabilities.
I think the next thing we should
probably check out the server.
Well in a university the working
language is English and then Mandarin
right.
Okay.
The university's name I think.
Oh that's okay. We have to clarify
because people keep people always come
of this question. the Chinese University
of Hong Kong.
We teach in English.
Okay.
Chinese means Chinese culture, Chinese
heritage, not Chinese language. Okay.
Right. That's what a lot of the exchange
students was confused about. They
thought like we teach in Chinese. No, we
teach in English.
Okay. Interesting. So, this is your
cluster.
Have you seen it before?
Uh in some other places.
Oh, but never his.
Yeah. Never here. Cool. Never in Hong
Kong.
Yeah, as I said, they they're being
full. They're quite well utilized. This
is actually not part of the cluster.
This is our noncluster stuff like web
server and stuff.
But uh yeah, so these two racks are the
cluster.
Did you have to do anything special for
the air conditioning?
So, I mean, as I said, this is kind of
like congestion, right? The thing is, uh
these GPUs are very hot, right? A single
8GPU system can put out like 4,000 W.
Oh, okay.
Right. So, it's like like a hair like
like two haird dryers. So when you say
when you say a single HGPU system uh how
many does each rack or each blade have
in it?
So like a 4U unit these days like these
ones like uh this one like and the and
this one these ones these are all HPU
systems.
Okay.
Right.
So for
uh is each of these all different
configurations?
So now our cluster was built up over
like almost 5 years. Right. So that's
why we we have some really old systems
that come from 5 years ago and we have
then we just keep adding things. That's
a bit different from let's say uh some
like uh the like say a full super port
which is purchase in one go then
everything will be the same.
Right.
Right. Here we just add keep adding
capacity and just uh leave the old
components until they gradually not work
until they're not useful anymore.
Not useful anymore.
Um
did you have to do special power design
power delivery? pull in.
Yes, you cannot just plug these uh GPUs
into a wall socket. It would blow the
fuse immediately. So, we we would have
we need uh well two free we have two
freephase power specially uh installed
in this room.
So, they have these massive plugs for
power. There's one there and then
there's one over here. And then you can
actually see they're running flex duct
work. And you can see it shaking at the
bottom of that one in the back where
it's actually shaking because it's
pulling in all the exhaust from the
servers. And they're piping that up. So
I think that's the return. That might be
the front side. We'll take a look. But
they're basically circulating it with
the flex duct work. Crazy. And that
looks like maybe more.
I think that's more AC going up to the
mini split up top. I guess how many you
know A100's or H100
right we we have something like like 30
A100
class GPUs and then we have a bunch of
like 3090s 3060s right so in total we're
talking about I don't know 50 or 60 GPUs
okay
right so yeah the the top of the line
GPUs are for teachers
what are the what are the 60 class cards
for like 30 60s
just teaching teaching it's for teaching
so uh we we have we have these uh
undergraduate machine learning courses
And we let students low hucking face
models.
These 3060s are for them. We don't want
them to occupy the A100.
And 3060s have 12 GB of memory. That's
more than enough to load some of the
smaller.
The good ones have 12. They made another
one.
Right. Right. Right. Right. We make sure
we get the 12, right? So yeah, that's so
that's sufficient for loading a small
model that
they can work on.
Right. Does memory bandwidth matter a
lot?
It matters. So to better utilize the
resources uh we offer the options of for
each researcher to load their own model
but we have also this centralized like
our own local chbg
like thing we have use open web UI
and with like uh with vom as a back end
so we we offer like just like this
preset
uh system where we can serve like germer
uh we can serve DC and the and when it
comes to getting high throughput uh yeah
bandwidth would matter
because for the export control the
government the US government does a
calculation right for some amount of
points like they made up a system
and uh memory bandwidth is one of the
components of the formula. Yeah,
they're getting smarter, right? Because
the band there has been multiple
versions of the band, right? The very
first band was targeting the
uh Envy link, I think. A the A100 to A00
N that has a little effect. It's a bit
slower,
but uh it's still okay, right? And then
I think they switched to targeting the
the flops.
Yeah.
But that I think is
there was a certain amount of flops you
could exceed. Then I think depends on
workload is I I think it didn't work as
well as they might have hoped for
because the H20s still still sold really
well. Well, I mean that's what's so
interesting about it is the governments,
at least the US government,
hasn't really caught up to what these
are, you know, like and so so they don't
know how to ban it, I feel like. So they
just kind of pick a number.
It's developing very fast, right?
Because if you go go back like 5 years
ago, everyone probably would think that
like, yeah, flops is a very important
metrics to consider. But turns out when
it comes to loading large models, what
we all care about is how much memory,
how much VRAM you have, right? you
simply cannot load a model if you don't
have the VRAM,
right?
So now H20 becomes a very attractive
option because
what's the spec on the H20 again? D.
So Hract H20 has drastically lower
flops,
right? But then it actually have it
actually has more memory than the
original H100.
What's the capacity?
I think it's 94 96
versus the 80.
Yeah,
the original version 80 is 80, right? So
it's actually in some sense more
attractive. Everything you guys have
named so far, it's all Nvidia.
So, are there any non Nvidia GPUs like
in the university cluster?
I think currently I didn't say any
nonvidia one, but I think it's promising
the AMD one. Currently, PyTorch also
still support AMD one. I think in the
future it will be another choice.
The Instinct Mi
AMD MI GPUs. Uh I didn't remember.
Yeah.
Yeah. We
What about uh Intel like the cuz they
just started making the B60s you know is
that interesting at all?
Like so uh I mean we we always like to
explore options right to have more
comput more computing power. We actually
expressed interest in AMD and we asked
around like to the outcoming suppliers
and they said that the risk is too high
for them
right because they have to support it
and uh they had I think there's one
supplier who told me they had tried to
sell an instinct
GPU in the past and then the and then
the customer returned it because uh they
couldn't figure out how to make it work
and then they realized that they don't
really want to carry the risk of having
a GPU that people are going to return to
so they would rather just not deal with.
Do you know any AMD GPUs that also are
export controlled?
I think the instinct line is also
covered under the cover export, right?
Yeah.
I mean, I guess that's also part of the
problem. If you're going to go through
the hassle of like of smuggling
something, you better smuggle something
that people actually want,
right? Yeah. And well, is there is there
any support for it? If it's if it's a
smuggled device
and you have a problem, what happens?
You know,
right? uh
like if a if a MOSFET blows up you know
so um now we obtain all these under like
like under following university
procedure and part of that is that uh
there should be warranty right okay
I mean now the the supplier is clearly
going to promise you warranty
right
but uh they also indicate that
now for the PCI version they can they
are pretty confident they can they can
support it because I think as they just
give then they just change it for a new
one and just Yeah. move try to move that
one out.
Yeah. One at a time. Right.
Right.
But if it's a HGX system,
even if they manage to get it in,
they don't promise anything basically.
Right. I mean for contractual reasons,
they will promise warranty,
right?
But uh whether they can actually do it,
they can't actually guarantee.
Okay.
Right. So actually for we have a legally
obtained uh HGX 800 system that we got
before the ban
and uh as far as I know after our
initial I think 5year support contract
uh ends
there's it's not possible to reuse
that's a 4U system with eight yeah eight
800 inside.
Do you ever physically go to pick up the
GPUs when you're arranging from a
supplier or they just ship it here? They
usually ship here.
So, it's just like to totally normal
commerce basically,
right? They just ship ship here by
It's so interesting because I feel like
when you think about like this concept
of uh like the black market GPUs and um
earlier I said I I found online a story
that talked about mi banana.
Oh. like and uh you think of it like
that, it it seems like when you go to
buy a GPU that's banned, it feels like
you would be going to a dark alley and
handing him cash, right? And he hands
you a GPU,
but that's not how it works.
Yeah.
No.
So, they just ship it.
They just I think it's good that way.
It's good for personal usage. It's not
quite good for a very big cluster.
Right.
Right. So, so does the if it's that sort
of easy where it just ships to you, does
the ban seem effective?
Like as I said as I said if it is
effective
in preending the building of a very big
cluster,
right? It's just not possible to get
100,000
okay
of these GPS
like we heard about some of the other
companies having.
Yeah. And I think I think I think there
are like people from the company who
recruited that idea. Right. We don't
really we don't really have that many. I
think that's
right. Right.
Why do you think the US government cares
so much? Like what is it they're trying
to prevent?
I I guess they just want to delay the
speed of other countries of training the
model.
But seems uh I don't know their ban is
very weird actually,
right?
Yeah. Do you feel like the ban is it
targeted at like military uses where
they're trying to prevent that and then
it just happens to affect you guys as
well or are they they targeting you
think just companies in general?
I I guess it's in general every company
every people will be influenced by this.
M
so I guess just delay the speed of
research just like like my experiment
for example if I can have better GPU I
can finish in one day but now two day or
three days that's fine just wait a
little bit
you get the same
I don't think
from the US government's perspective
there's this distinction between like
Chinese academia versus Chinese military
versus Chinese commercial I think no I
don't think they're they really consider
There's a distinction, right?
Yeah.
Just total
Yeah. very general band,
right?
Is um I know Huawei has the new Ascend,
I think it's called.
Is that interesting at all? I mean, does
it seem like that's promising for you
guys?
Uh Huawei Huawei GPU I think they built
that for many years and they also
trended some length model based on that.
So currently seems uh no no friends
around me are using but maybe in the
future they are trying to try that if
the they still cannot get the Nvidia
one.
Do you guys need CUDA for your stuff?
Oh yeah definitely if you want to train
the model you need CUDA
right? So I feel like uh we can still
get by because need we as I said we're
not really needing that many GPUs right?
So there's still enough supply Nvidia
GPUs
supply around for what we need. Right.
Right. But uh
if we want to scale up then Nvidia is
just not possible
right now.
So probably one way eventually like
something like Huawei Ascendants would
be.
You said you have a an interesting GPU
that was modded I guess. Yes,
we have several GPUs like like as I said
like this is one of those like u
reference board.
Yeah.
Uh and
uh Audi's 3090.
Yeah, this is a 3090.
There's a 30 This is a 3090.
I think it's very Chinese specific,
right?
There I I don't think you can actually
They were not popular in the US. You
could get them, but not many.
Yeah, but because they're noisy. They're
really noisy. And the only reason to do
to get these are because you want like
multiple of them. Yeah. Right. Packed
closely together. This designed for that
purpose, right? So they have these kind
of slots. One of these I think are like
a normal actual PCIe version of A100.
And this one is actually a molded
version, but they use the same casing.
Okay.
So this one is actually molded from a
SXM version. So if you
if you install it and take a look at the
uh
Nvidia uh like uh system information,
this one will show up just like this
show as SXM.
SXM
SXM, right? But they look the same. Um
this is actually a very interesting
question because as far as I understand
these this is possible because they
these these GPUs are assembled in China,
right?
These GPUs are assembled in China. So
they apparently I guess they have spare
casings, right?
Well, so here's a question for you.
If uh if a band GPU is assembled in
China,
then what's to stop them from just
taking it?
That's actually very good question
because um as far as I know, even the
H200
seems to be assembled in China.
Yeah.
And I have no idea what's preventing
these GPUs from
falling off of the line.
Line. Exactly. So, yeah,
I don't understand how it works at all.
How is the band even going to how is
going to ban is even working like the
box of these GPUs at least state they
are manufactured in China
right
right so I'm not sure how actually that
works
I mean it's yeah it's like
there Nvidia is not allowed to sell it
to China
but they can make it here
China
and then the intent I guess is they ship
it somewhere else and then they can't
ship it back
right
but it just seems like if I'm the
factory it seems like there's a lot of
money And so
we had some QC defects. Right.
Right. So my guess is
they must be spares. I don't know.
Right. Spare spare SXM like modules,
spare casings and then somehow these
spare parts just get assembled into a
complete GPU and get sold.
Yeah.
So currently it seems like Yeah. Like
that that this whole GPU manufacturing
thing has the flavor of that, right? the
all these high-end GPUs are still
manufactured in China, but they're not
allowed to be sold in China.
Yeah.
And somehow the US government thinks
that's going to work and somehow the
Chinese government also allows that to
happen.
I have no idea how actually that whole
thing works,
right? That's so strange cuz like the
GPU silicon is made in TSMC, which is
all over Taiwan mostly. Yeah. So, they
make it there, they ship it to the
factory in China, I guess, and then an
SMT line must populate the board. Right.
One of the GPUs we have, one of the
A100s we have that are molded. What? Um
I think uh the N one of the Ning doesn't
work.
So my guess the my Yeah. The explan the
explanation is is probably
failed
failed PC. Right.
And then it just gets assembled and
Right. So
and so then you have two out of three I
guess NV.
Yeah. So yeah, you just have Yeah. So
what? Yeah. So it's just going to be
slower.
Right.
Right. But for a bargain price
Yeah.
Right. Is it cheaper than uh like a
fully functioning?
Yeah, like I think uh I think it Yeah,
it was cheaper. Okay. It was cheaper,
but uh doesn't make a difference if
you're just using one GPU,
right?
So that would be a So it would be a
bogging.
Interesting. Yeah. Do you think Nvidia
knows that all of this is happening?
I would be surprised if they don't,
right?
Yeah.
I'll be surprised. I would be really
surprised if they don't.
I see.
Right. These are very expensive items.
Yeah.
Right. I would imagine you would keep
track of everything, right?
Yeah. I'm not I don't know. Is that
I'm Yeah, it's hard to know how what
they going to do, how what what they
plan to do with all these uh defective
parts.
But I'll be very surprised that no one
have ever thought of the possibility
that if it's something so valuable,
someone would come up with a use of
even a defective one.
Defective ones.
Yeah.
Yeah. Interesting. Yeah.
Yeah. Like uh the AG like the AG GPU
servers like we could pull out one of
them if you want if you want to see
what's
if it doesn't disrupt you. That would be
great.
One of them.
We have another
pull out one cuz it seems interesting.
So
So this is one of our older ones. this
older one because um this one is not
it's assem it's basically assembled
completely from like retail parts.
Okay.
Right. So
this whole system is around obtained
around uh when the time when chachi
beauty came out. Right. Yeah.
So this is the time when this
everything you can get from AliExpress,
right? The case,
right?
So you bought all of this just on
AliExpress.
Yes. Everything is from AliExpress.
everything express
like CPU, RAM,
this the by the hardest by far the
hardest to get is the power supply.
I saw you write about that in your blog,
right? Um what makes it difficult?
Because now it's very hard to get high
quality high power
server grade power supply. this like you
need in order to support four H uh 4
A100 you need like uh 2000 2,000 watt
power supply and you need you need these
kind of server grade redundant power
supply and there's just no demand
in the re from a retail customers for
these kind of GPUs so for these kind of
power supplies this is like this right
if you buy this case
mostly they come with something like a
800 watt power supplies
and we need 2,000 so that's the hardest
part Right. Resource.
Is this a redundant power supply?
This is a redundant power supply.
Okay. So, it's got a fail over the CPU.
What are you using for this?
This is a uh Epic 776. Epic 776
that you can get from AliExpress. No
wonder if you need to. Um and then the
board is from Srock.
Yeah.
ASRock board.
As rock. So, it's a like a server line
of does have like seven PCIe slot. Ah.
But then in order to because we need one
for we want um infinite band network.
Yeah.
So we have these cables like specially
made to the
Oh custom made
to the link we need.
Okay.
Right. We need it to be this long and to
be to support the infinite band card.
Is that that down there?
Yeah. This is the like
So that's the infinite band card.
Right. That's a 200 that's a HDR 200 GB
card. Yeah.
Is is all this stuff uh pretty high
priority for the university at this
point like on the I mean technology side
they are they invested in the purchasing
and you know
I think uh I think the university is
realizing how important this is right
that's why we have this brainstorming
session
right
yeah I guess university will begin to
buy more GPU and try to find some
strategy to share with the whole staff
members
right
yeah it's a good sign
cool
but it's just a start it's Right. It's
Yeah. Like uh I think we would try to
catch up quickly.
Yeah,
we try to catch up quickly.
Very cool server room. It's impressive.
Why are they just happily providing with
their way? And uh I mean I'm not
encouraging working 24/7, but like
customer support in China is 24/7,
right?
Not designed for Hong Kong.
Designed for Hong Kong,
right?
They don't sell it anymore.
So Hong Kong is a free trade port I
guess. Yeah.
But China mainland is not.
Except Hainan, right? Except H.
Yeah, Hainan and some part of Shanghai
also
makes similar policy.
And that just means no uh local tariffs,
no import taxes.
Yeah.
Right. Right. And as very as relatively
simple paperwork, I guess.
Okay. All
right. This is where we call the
university mall. It's where we hold our
like a like a commencement.
Cool. Yeah.
The graduation ceremony in here.
Yeah.
Wow. This is nice.
Yeah.
Holy [\h__\h]
So, that's the science park of Hong
Kong. And you actually see one of the
our unicorns
uh a little bit. Sense time.
Sense time. What do they do?
Uh facial recognition. Preveris. Yeah.
Yeah. I would say AI,
AI, AI, but uh they they they they they
kind of built their business on on
facial recognition,
CCTV stuff.
Chinese government is a big is a big
customer,
right? Do you feel like you guys are
locked in at this point to Nvidia
because of CUDA? Like does I guess to
rephrase it,
is there a difference in what matters
more to you? You know, the power of the
hardware or the power of CUDA cuz both
are very good.
So
I would imagine if someone put in the
enough effort so that non Nvidia
hardware
runs just as well as Nvidia hardware in
PyTorch.
Uhhuh.
And uh then the VOM and stuff then uh
would not be too hard for us to switch.
But I guess that's harder to do than to
say.
Yeah.
Right. Because someone has to put put in
the effort to make the models run
smoothly.
There has to be a good reason to switch.
Yeah. That part that part is hard.
Yeah. Only problems is if you switch to
other devices, you will have a lot of
bugs,
right?
Some bugs you cannot fix. You need the
whole community to help you. So, this is
a very slow procedure.
Definitely the CUDA and the Invadia
currently is the best one you can
choose.
Right.
After this discussion, Zuha parted ways
and Vinci brought us to one of his
favorite tech spots in Hong Kong, the
Golden Computer Center and the outdoor
tech flea market next to it. We actually
have a whole separate video looking at
some of these places.
After
this trip, we asked him if we could go
find some supposedly banned GPUs like
the RTX 1590, except available in public
where in theory you shouldn't be able to
find it. The hope was to find a
shopkeeper with some ground level
knowledge or consumer level.
Everyone wants Jensen's flavor.
And you do you still frequently go to
the market?
Not as often as before.
Right. So mostly because I've mostly
switched to using server components,
right? And you can't you can't get these
components in these markets. Our cluster
is pretty much exclusively epic, which
is rare when we start building out 5
years ago.
5 years ago, there's probably still a
lot of Intel,
right? Everyone is still buying Intel
and we started buying AMD. So right now
in a on in a cluster uh we have the last
generation Gino CPUs and I think yeah we
have 256 score across two CPUs I my
guess is these GPUs are probably
smuggled in from either yeah from
Southeast Asia in that case unless those
countries try to enforce
right
the ban
but I feel like most of them probably
wouldn't really care you know it's not
it's not their ban
right it's not their banana Uh
why why why cross the Chinese,
right? That's a good point, too.
Yeah. Yeah. I guess they've got two
choices, right?
Right. Which is why Biden
piss off America or off China.
Right. That's why Biden had this right
idea where the Biden administration had
this idea where you're going to have to
separate the world into different zones
and
some zones are trusted and some are not.
Ah,
right. But then this this is scraped,
right? this. So now it's back to I guess
back to one by one
by one. Yeah, back to one by one. Right.
And I I I haven't heard anything about
like the countries having to prove
themselves to be trustworthy anymore.
It must be frustrating for Nvidia on
some level cuz
I'm sure they would ra they would they
would rather much that they can just
sell what whatever they want to China.
If they come up with a product that is
in high demand that means it's useful
for AI and if it's useful for AI the US
government wants to ban it.
All right.
Do you think the export controls
are having their desired effect
like for the for the US?
Yeah. Well, yes, in terms of stopping
China from building a comparable GPU
cluster to US,
right?
Right.
So, it's really just about what you keep
coming back to, which is if you want to
buy thousands of them,
it does make it a lot harder.
Because I guess ideally if you want to
buy 10,000 GPUs, you probably buy it
straight from Nvidia, you can.
Not even that many. Even if you're
buying like say 400.
Okay.
All right. Like let's say a super port,
right?
Then the Nvidia is likely going to be
directly involved. I do wonder,
you know, it seems like if you make it
hard enough to get
US products, um
eventually
you would think China would just dump
its own resources into helping like a
domestic, you know.
So, well, China is already doing that. I
guess China Chinese government is trying
very hard to come up with
competition,
a competitive GPU.
All right, so where are we going? What's
this place called?
This is uh called This is This is a
shopping mall called Dragon Center.
Watch your belongings.
I did lock all the doors, right? That's
just me. I mean, okay. I asked him
if it's safe to leave it in the car and
the first time we say it see belongings
from the police.
I mean they parked they're priceless
too. I pretty transparent.
Oh this one it's a parallel import.
Parallel import.
Where is it?
Oh. Oh
parallel import.
What does that mean?
That means not imported by an official
distributor. So like kind of uh
what rate it practically means is
smuggling.
Smuggling.
Yeah.
Right.
So what um what are their prices?
So the cheapest one
cheapest one says it's a silk.
That's a big price difference.
Yeah.
Almost 2x.
But the ROG ones are always expensive.
Yeah. But they it's not worth that much,
you know, like the the price difference.
I mean
I guess when it supplies one.
Sure. Yeah, one that someone maybe
brought in from another country. I guess
that's interesting too, like if they
have to account for cost of like plane
tickets and stuff.
So that astral is $5,223
US.
So it's the cost of even it's the cost
of like almost two 5090s at home for us.
It's even in stock. The cheap one the
cheapest one is I think it's about 3,000
24. So yeah, 3,57 USD. I mean, that's
not like that's pretty close to the
average price in the the US. The average
price is 3,50 right now. It's actually
like the same. I see they have D.
The 5080, I guess, is still okay. It's
not band, right? So,
yeah, but I think it's, if I remember
correctly, the 5090D uh wasn't that
expensive.
So, it's probably the price has gone up,
right?
Yeah, cuz I mean it's like more
expensive than the cheapest 590.
Yeah.
Interesting.
It's possible that um
it still comes with warranty.
Mhm.
Like these ones probably have no
warranty at all. Oh, no. I mean 3 years
Hong Kong warranty. I think what that
means is that the store promised that
they're going to help you,
right?
But I think these ones that they were
imported when back when it's legal,
right?
So they still I think they actually have
official warranty.
That's actually Yeah, that's a good
point. I wonder if you bought one when
it was legal if you still get support.
Yeah, I think that's the case.
Yeah,
at least that's the case with our data
center GPUs.
Okay, that makes sense.
Oh, there's a D. The D is uh banned now.
Yes, they can come up with DD or D2.
Yeah,
I think the price has gone up in like
now that Yeah, it's it's going to be
better.
Yeah, it's interesting. You can buy a
5090 FE here.
Yeah.
I mean, but it's supposed to be banned,
right?
Yes.
But you can just buy at retail in a in a
store.
As I said, it's not illegal to buy them,
right?
But I guess what surprised me is how
easy it is to buy them.
The right price.
36,000 HKD.
That's pretty expensive.
That sounds expensive.
That's like
4,000 over $4,000.
$4,600 US. Wow.
See?
Yeah. But you can get it there. So I
guess the main impact then of a band is
the price goes up,
right?
Yeah.
Any idea how they get their 5090s?
Same way you do.
Same way how the how the high-end GPUs
get moved in.
Okay.
Although I think for these ones there's
a chance that uh it's actually directly
moved in from other countries.
You think uh one by one?
One by one.
I think I I I've heard that some um
students
would do that.
Okay. like, you know, buy one from Best
Buy,
right?
Move it to move it back
to a third country.
No, just straight back to
straight back. Okay.
Straight back.
I mean, if if it's with a premium like
that,
do you happen to know is Yeah. Well,
yeah. I mean, that premium is great.
They can double their money, right,
is there um do you happen to know is
there any law against bringing it in for
personal use
in Hong Kong? No. I mean like from the
US any like if customs stops you, you
know, but I guess they would have to
find it in your bag. So they'd have to
search your bags. It's pretty unlikely.
Yeah.
And you would you you're relying on the
custom officers knowing that this
particular moto
Good point. Yeah,
that is a good point. And then caring
caring
like if Yeah. Okay.
I really don't think that's going to
happen,
right? I guess that's how they get in
one by one then.
I move forward.
Uh probably 550s uh battery in China.
What's that?
Uh we we only get a uh 1590 uh stock.
So the 1590 the founders edition maybe
not in stock.
Yeah. We direct directly order from
other countries.
What's the most popular card?
Uh 90 no uh always come from Australia
and uh the second one is Thailand. Oh,
so the 90 normally comes from Australia
and Taiwan.
Yeah.
Can you buy
bulk a lot of them at once or is it just
one at a time?
Uh very tricky is that uh almost you can
order from mainland China.
Oh
yeah, you can buy a lot.
Oh, mainland for 90.
Yeah.
Okay.
Recently we got the uh app from Facebook
that are selling in monk. It will
provide lots of uh 90.
Okay.
Is it expensive?
Uh
very interesting. Uh 90 only uh
sometimes cheaper than 90D.
Okay. Wow. Do you guys ever sell like
RTX 6000 or is it too
uh because RT 6000 we got ADA edition?
ADA. Yeah.
Yeah. ADA edition. So uh replacement
will be a edition
but no blackwell.
Uh yeah otherwise uh also from uh magn
right. Okay
interesting 5090. Do you have to get it
from a different supplier than 80?
Some models yes some model no.
Okay.
Because uh the cut problems uh the
distributor only can send back to the
original page.
Got it.
Yeah. But a local uh distributor uh they
can do DOA or IMA.
Okay.
Yeah. Because uh the US problem Yeah. Uh
the like uh 4090 or 590, right?
If they got problems, they cannot get
back to uh China.
Do you do you have any idea how
uh the mainland suppliers get the 5090s?
No idea.
Yeah. Because uh too much supplier.
Yeah. They are doing trading they can uh
transfer the other regions to Hong Kong.
Okay.
Yeah. So we have no idea what way they
come but we can know uh which region
like Australia, Taiwan. Thailand get
lots of uh stuff.
Yeah. in in because uh like last
generation like 40 90
Yeah. the price risk very high. Yeah.
But uh I don't think that in uh 50
generation they can make lots of offers,
right?
Yeah. But uh it's not seems like uh what
they thought.
It is still funny to me just like the
how how much it flies in the face of US
export control that you can just walk
around in a public facing mall and buy a
5090. Now given that it is very
expensive, right?
But it's not like you just can't buy it
anymore. So, I guess if you really need
it, you can get it.
And I really don't think locals think
much about buying the GPU.
GPU is added in price, right? It's
expensive. But, uh, the fact that it's
there's an export, I don't think cross
the mind.
They don't really consider it.
Yeah.
With that, we wrapped our first full
tour of Hong Kong. The city was
impressive and filled with character and
culture and GPUs apparently because one
of our next stops sent a message the
night we were planning to hitch a ferry
to Shuko port in Shenzhen. He told me
they had just received a batch of GPUs
to their Hong Kong warehouse and that
they were in route to their Shenzhen
facility and that we could buy a GPU as
soon as tomorrow as long as we wired him
money immediately. That seems like the
responsible thing to do. And because
it's fully above board as a buyer and
because he also texted me a photo of an
RTX Pro 6000, which was very intriguing
since it's Blackwell, we decided the
best way to learn about buying export
controlled GPUs in China would be to
just do it. So far, all the contacts we
spoke with were aware that some form of
factory repurposing, theft from the
line, QC reject repurposing, repair, and
actual byhand smuggling are involved.
But none of them knew for sure how the
GPUs actually move. To get closer to the
source of the movement, we decided to
wire those funds and get an address. You
there,
Stephen? Yes.
Stephen?
Yeah. How's it going?
Oh my god.
Um,
all right. You really help here?
Yeah. So, I've got $3,289
going out.
Perfect.
Yeah, that'll be from the the GN account
and then it's going to and their bank
and then account number is uh Swift, I
guess, is what you need probably. Um.
Mhm. Okay. Good to go. I will get this
done for you.
Perfect. Okay. Thank you.
All right. Have fun. Be safe.
Thanks. See you.
Bye.
We'll see if the money's gone in 2 days.
So,
that'll be that'll be our next
adventure. We packed the hotel, said
goodbye to the famous Hong Kong junk
boat, potentially said goodbye to
$3,000, and boarded a ferry to Shilo
Port. But we're going to take a quick
detour in the story here because we have
a source who says he has more
information on how GPUs get in to China.
So before checking with our GPU
middleman and our fence, we're first
going to meet up with someone else.
Next up, we took a couple hour car ride
to Hujo to meet with a guy known as Mr.
Five. Hujo is a city with some serious
grit.
The city has grown to take some of
Shenzhen's factory industry as it's been
pushed out over the years with
Shenzhen's expansion and as it's turned
into more of a metropolis. This is a
complete 180 from the finance hub that
we were just in. We've spent a good
amount of time at Hujo over the last
decade, mostly visiting case, painting,
tempered glass, bending, and tooling
factories alongside those for heat sinks
like video cards. He's a cooling
hardware reviewer and has specialist
knowledge in factories that make video
card cooling solutions, including
Nvidas. This experience allows him more
access to information about the peculiar
relationship between the factories
making the cards and the companies that
on paper can't sell them to the country
where they're made. We instantly related
to Mr. five for his own editorial
dispute with Nvidia's review sampling
process where we've both felt that the
company seeks to control review
direction. His username is 51972 on
Billy Billy. Even if you can't speak
Chinese, you should check it out cuz
he's got charts and everyone can read
those. Most of this interview is in
Chinese, but we've transcribed it in
subtitles and there are English sections
as well. The next interviews will be a
mix of both languages. Okay. So we are
joined by five that is that is his
English name. I noticed when we came to
China in SEG market, I noticed a lot of
5090s.
Mhm.
Do you think most of the
band GPUs come in one at a time?
Uh
GPU.
Mhm.
Why didn't you call
Fore
MSP. foreign.
Made in China.
Mhm. It seems a little bit weird that
these products are made in China.
They're made here, but technically
they're not allowed to be sold to here.
champion.
Um,
so essentially um, you know, China is
the the lead manufacturer for thermal
solutions, but once you get into
politics, it's just something where you
don't have much control,
right? And given the entire global
political like situation,
yeah, it's just this is not a surprise
anymore. So even though on on a logic
standpoint doesn't really make a lot of
sense, but
it's politics.
Do you do you think Nvidia knows about
this?
Nvidia's
like you turn a blind eye.
Okay.
Yeah. Just turn a blind eye.
Okay.
It's too much money to be made. So,
they're they're not going to address it
directly or acknowledge it, but they
know what's going on.
Do you think Nvidia uh would want to
stop it?
Boy,
no.
No way.
Okay. Uh, okay. Cool.
Viewers,
where should they go to find his
channel?
51972
five.
This was an interesting conversation.
First of all, I learned a new idiom.
Secondly, we turned it into a t-shirt.
So, if you're liking this content so
far, please take this brief intermission
to head over to store.gamers nexus.net
and grab our blind eye t-shirt to
directly support this content if you
like it. Long form content is hard to do
on YouTube and we can do it because of
your support. We try to make some pretty
damn cool stuff to get you something
useful in return as well. And if you're
watching this sometime close to when we
publish, we've still got the digital
reward tiers available for our backer
tiers for this project. We really want
to produce more stuff like this. But
from our point of view, we need to earn
that. If you think we have, head over to
store.gamersex.net
and grab one of our new blind eye
t-shirts, black market sticker packs,
GPU VRAMm glassware, or even our GPU
shredder t-shirt for the paper launch.
Thanks for the intermission. Let's get
back to it. Up next, Shenzhen.
Our first stop will be in Shenzhen,
Baoan to meet with a trading company
that sits between Hong Kong and Shenzhen
warehouses. They're the trading
company's trading company, and we're
buying an RTX 1590 from them, if they
exist, hopefully.
Shenzhen was a fishing village just 40
years ago. Now it's one of the most
technologically advanced cities on
Earth, and it's grown unbelievably fast.
Shenzhen got growth from pushing farming
and fishing out and bringing factories
in. Now it's even starting to push some
of the factories into the outskirts.
Shenzhen has some extreme cyberpunk
vibes with its mix of technology and
surveillance. It has packed tech markets
in Ha Champ, including some of the
weirdest computer parts we've ever seen
in our prior visits. There are entire
buildings dedicated only to small phone
repair shops with their own kiosks.
Another building dedicated to a mix of
computer hardware, gaming, and
miscellaneous components. More still,
for just integrated circuits, literally
an entire building where people pretty
much only sell things like capacitors
and MOSFETs. And all of these places
have people who know people. That's
their job. Shenzhen's technological rise
comes with it an uncomfortable
omnipresence of CCTV and facial
recognition, which feels more fitting
today than ever before now that we know
worldwide government facial recognition
is a big user of AIG GPUs. We started
the day by meeting up with our
translator for the next 2 days, Raymond,
who's helped us on factory tours for
years. Because Uber doesn't work in
China since it uses the Google Maps API
and Google is blocked by the firewall,
Raymond was also our man with the
locally compatible apps to get us
places. This became especially important
now that even cash is becoming less
acceptable by some cafes and
restaurants. Everyone's moved to paying
with the WeChat app. With our ride
booked, we're off to see if the wired
funds turn into a GPU.
So, we're going to go meet our uh
GPU supplier.
These guys seem to understand that we're
media and we wanted to film, but we
weren't 100% sure. So, we walked in with
the cameras off and once we established
that, we proceeded.
Like, would would one of them want to
talk with me about just buying and
selling GPUs or
just can't show their face? Oh, okay.
Yeah. Our company name.
Oh, okay. So, you don't want
Okay. So, we can talk about it, but
don't show face or name, I guess.
Yeah.
Okay.
Face or name or company.
Okay. But talking about it, it's fine.
Talk about it is fine. Yeah.
Okay. This is the card. I guess this is
the one that
Yes. that I bought. Okay. Can we record
this?
Yes. Yes.
Okay. Let's do that.
Yes.
Cool.
Um, from the Hong Kong warehouse, what's
the most common GPU your customers buy?
Bye-bye.
Black uh GB GB GB GB
GB
Mayo.
Okay. how how popular like 5090 58 or
5090 5090 uh
you know class cards versus the AI H100
H200
I heard the 5090D is Like Nvidia start
making it.
You sent me the RTX6000.
Okay.
and they have it in their you basically
um buy them buy the GPUs from a
distributor.
You mean the agent?
Yeah.
Oh, okay.
Yeah. Everything is really deep.
Yeah. RTX 6000 just won the shipping
196 GB, right?
Yeah. 96. Yeah. Uh, what's the price in
your country?
Yeah.
Do you need to pay the tax fee for?
Yeah.
So, I guess is is the most common user
like AI companies, robotics companies?
They don't really sell to users, more
like trading companies.
They don't usually get this type of
inquiry like single.
Yeah, we had talked about GPUs for like
40 minutes. At this point, I guess they
got bored and they started asking us
questions, but we'll just include it cuz
it's kind of a fun culture exchange.
Yeah.
um does anyone buy AMD AI GPUs or AMD
like highend
mi
Oh,
c u dydo
intern
behu
Be bey
promise
mayo. There's no restriction.
Correct. Yeah. Mayo.
They said that uh Nvidia falls into
distributors I guess and then they get
distributed into here. Okay. Do you
think do you think Nvidia
knows about people uh buying and selling
the band GPUs in China?
Like are they aware of that? Do they try
to stop it?
They know, but they
they're like, "Ah, you guys can sell it,
but but
but we don't want to know about it."
But yeah, we don't want to like easy and
easy.
Funny. I wonder how many people they see
walk out just with a GPU. They
definitely shared more than I thought
they would,
so that was good.
That's good. But yeah, I it also
connects the pieces really well with the
professor we spoke to.
Okay.
Cuz that guy, he shared like everything
new.
Yeah, I guess now we probably just go
basically back to uh Hua Chambe is a
little bit north of that. Uh so our next
stop is 10 minutes away from the hotel.
Okay,
it's a pretty far ride back. Um, the
next one is even more
uh unknown if they'll talk to us
because you didn't buy anything from
them.
I didn't buy anything.
Um,
and I didn't I didn't offer either and I
explained the whole situation. Just
replied to every single message with
okay, okay, okay,
except for except for um the address. He
said, "I'm not sure what to talk about,
but I can try."
Okay.
Which is like like maybe a good sign.
I don't know.
I'm not sure.
Hard to guide, right? Until you see him.
Yeah.
So, we'll try that. Um I guess we can
keep the camera out in the car and then
we'll just probably DD back.
I think you want to try it?
Yeah, I'll try it.
We're not 100% sure where we're going,
but we'll get there. So, I think our
biggest takeaway is uh
these GPUs are banned for sale into
China, and I just walked into an office
and bought one. So,
the the efficacy of the ban is maybe
questionable. This still tracks with
what Dr. Da Vinci was telling us, which
is that it appears to be the difficulty
comes from buying lots and lots of them.
But even the people we just bought this
from, they're like, "Yeah, I mean, if
you want more, just ask and we'll figure
it out." So the whole concept of this is
a thing you cannot own or purchase in
China, uh, it really just doesn't appear
to be as strictly true or simple as we
might otherwise think. From the surface
of an export controller, a band, you'd
think you just can't get it. That's not
the case. You can get it, and it's not
even that much more expensive than what
I would have paid in the US. In fact,
their RTX 6000 Pros are the same price.
They're actually cheaper after taxes in
the US if you buy it here. And that's
also export controlled. So, that's our
uh that was our adventure getting a GPU.
Move on to the next one. The most
important thing we learned here was
where to go next. Just like the ants
moving the GPUs piece by piece, we'll
have to collect our information piece by
piece. Recapping the parts of the
hourlong conversation more efficiently
and in one language, the company said
that some cards have become more
difficult to get, but they can still get
them. And more expensive or not, it is
the reason that their job exists. The
fact that they texted me a photo of the
brand new at the time RTX 6000 Pro
Blackwell card, the very same one that
we just bought for $8,500 in the US and
that was hard to get at home, shows that
this company and their distributor are
resourceful. Their price is $8,600 US,
which is actually cheaper when factoring
in US taxes and shipping. So, the ban
isn't stopping them. And the 6000 Pro is
a serious AI card with 96 GB of memory.
It's banned and it is definitely in
demand as well. This company isn't used
to a single GPU sale like ours today. It
mostly transacts with other trading
companies in high volume. They
interfaced with a Hong Kong distributor
to bring the cards into Shenzhen, moving
it across one more border, another ant
in the chain. That meant they could
connect us with their distributor, which
would be familiar, we were told, with
where we could find smugglers to talk
to. But this company itself neither knew
many smugglers directly nor knew many
end users. And that's okay because each
link in this chain will get us one ant
closer to the information we need. So,
we took note of their information on the
distributor and the smuggling side to
use when we got back home. Seems like a
conversation to not start until after US
customs. And then they helped point us
toward our next stop. Up next, Hua
Champ.
We hopped in a car and drove 40 minutes
to meet up with a GPU trader we found in
Shenzhen. Huh. Champ is home to the
world famous SEG market or psycha where
we've found some of the strangest
computer parts we've ever seen in past
videos. Qua Champ has the highest
concentration of technology and
integrated circuits and electronics in
the world with a neverending maze of
multiple disconnected multi-story malls
specializing in all electronics. If your
life depended on getting a complete
product made in a single city block,
Shenzhen Hua Chan Bay is your best bet.
Fortunately, this time we'll be with
this guy. This is Vincent. His profile
picture on a messaging app is Van Gogh,
so he has a sense of humor. This guy
wheels and deals with dozens of GPU
trading companies near the bustling Bay
Plaza. He has a reputation around here
for being the smooth talker.
Our
first impression was that this guy knows
people and they already know him and
that's what we need. Vincent generally
seems to have an attitude of getting
[\h__\h] done. But before we met him in
person, we had some concerns going into
this that it would be fruitless or that
no one would even be there to meet us.
All right, so second location to try and
talk about GPS for this one. No idea how
this is going to go. I've been talking
with a seller back and forth for a
couple weeks and every single response
is is two letters. Okay. And so I don't
know what that means. Raymond is going
to try and explain the concepts.
Okay.
See see what?
All right. Let's go.
It looks like the right place is like um
a lot of tech companies. Yeah,
we had a lot of trouble finding our guy,
but fortunately it wasn't long before he
found us.
Yeah.
He's pretty open about it.
Okay.
Whatever he knows.
Is it difficult to get the GPUs?
Like there's a lot of
atg.
So does he like just buy and sell? Is he
a trading company I guess?
Yeah, he's a trading company.
Trading company.
So mostly from Yeah.
How do the GPUs normally get here? uh
like what's what's the process normally
for them to actually cuz
highend GPU highend
GPU
Sorry. Taiwan,
one by one.
Oh, they do more piece by piece.
Yeah.
Yeah. From
Hong Kong.
Yeah.
Is there any consequence for bringing
them in? foreign.
Okay.
Does China care or does Hong Kong care?
Oh, so like in China is it's not it's
not like a real restriction, right? It's
more from the US side,
right? Right.
Yeah. So,
Everything he says you can buy from the
shops, right?
Yeah. Like he doesn't exactly know
where, but like Hong Kong is like the
most closest and most common.
Yeah.
That is easiest to
bring it in.
What's the normal like order amount?
Like how many how many cards does a
company normally buy?
One or two single units
like a
Don't
It's early.
Speech. Speech. Speech.
Okay.
Don't want to
Yeah. Okay.
Michael Jordan.
Does he need to make any trips to SCG to
buy stuff that we could potentially go
with him for?
You know what I mean? like to like if he
has any need to go to Saiga to buy
stuff, would he be interested in us
following him
to accomp like accompany him?
If you could find a soft way to present
that, I guess.
Oh, okay.
Okay. I guess we see like some of his
favorite spots or spots he thinks are
the most interesting.
Yeah. Yeah. Okay, cool. Yeah, that would
be cool.
Okay.
Is it walking distance?
Yeah. Yeah.
Oh, is it really?
Oh.
Vincent led us over to various large
wholesale and consumer retail markets.
His office is about 10 minutes away from
SCG Market, which is his daily haunt.
There's one peculiar detail in all of
this. Vincent has customers both in and
outside of China, but his foreign
customer base is relatively high. He
mostly deals in RTX490s, which seems to
be a trend, as you'll see with our
smuggler contact at the end of this.
That means Vincent is buying 4090s that
either never left China, but were
supposed to, or were likely illegally
exported to China, which would mean he
is then re-reexporting them to his
foreign customers. It's like an infinite
loop. He's an interesting guy. He talked
about multiple business ventures in
Uganda and Kenya.
He said he went to Kenya before.
She's saying he had a business there
before.
Oh, he wanted to go there and uh start a
business there. So, he was like looking
around and see if there's any
opportunity.
Oh, he's like the the people couldn't
afford it. Like the things that he
wanted to sell was out of the the market
range.
Okay. Too expensive for the locals.
Yeah.
He used to work as a shopkeeper in the
mall itself and would interface with
factories or anyone else who wandered
in.
Just talking about his business.
Yeah.
Yeah. It's pretty simple to set up a
business or
have start running a company like his,
right? So
low cost and then uh
it's like also like very scattered.
He used to own a shop in there.
Oh, really? Yeah.
Okay.
Oh, yeah.
Okay. We can You can put it down to the
side.
Did they let you do it last time?
Uh, we snuck around them last time.
Yeah, we looked for them and then we
went around them.
Oh, wow. These days, he's closed shop
and is a buyer instead. He mostly takes
orders online, doesn't keep much
inventory, and just walks across the
street when an order comes in that day.
It's basically like a concierge GPU
pickup service.
Okay.
Cuz you can just come here and pick up
your stuff and then once you get an
order and just,
you know, once once you get the order
online, then you pick up the stuff here
and sell it. So he doesn't have to
actually put down any of his own money
to buy keep stock.
Okay.
This I think I've been to before.
minor
minor. Okay.
Vincent did what we soon learned he does
best, which is connect us with people.
He brought us to a friend of his who
sells Bitcoin mining hardware and GPUs.
This is where we learned the
relationship. She thought he was
bringing us here as customers, so she
didn't want to share his cost with us.
She said it's up to him to tell us what
the price is and then she'll tell him
the cost.
Okay.
Yeah. A1000.
What?
A1000.
Where is that?
Right here.
Hello.
19,000
800.
That's not bad, honestly. Like, that's
pretty pretty close to US pricing.
2,600.
The average price for 5090 right now in
the US is $3,54.
It's actually like cheaper than average.
But
that's what the other guy was saying are
like the prices is like so similar.
BTC the
Okay, the
like with other interviews, we learned
that people rarely want AMD and
basically nobody wants Intel.
AMD.
Is she one of his suppliers? Do you
know?
She's like he does what he does. They're
very similar.
Okay.
He knows her, I guess.
Yeah.
Okay.
cuz he started off as like doing mining
machines, so she's still doing it.
We told Vincent that we were getting
ready to travel 6 hours north to check
out a GPU repair shop that buys small
components for not only repairing broken
video cards to keep them in service,
especially the expensive band ones, but
also modifying and building new video
cards that Nvidia doesn't make. He said
he had an idea for us.
Nothing special, but we can go check it
out.
Yeah. Okay.
Protect peace.
Yes. Yes. Yes. Yes.
Protect peace.
That's my That's my translation. I don't
know if it's right.
It is. It is. Exactly.
That's like finished goods. And this is
like preassembled.
Okay.
This is it.
Cool. Wow.
I don't know what I'm looking at.
The ceiling just says New Asian.
I don't think I've ever been in here
before. This is cool.
Yeah.
Oh, MLCC's capacitors.
You find those on GPUs everywhere.
SMD service mount device dip.
Wow.
I see.
Okay.
Okay, cool.
So, he's they're relatives.
Yeah.
Okay. Cool.
Well, is one kind of relative.
No, they are selling on like across
different platforms like so it's more
like e-commerce.
Okay. Do they sell on Alibaba also?
Alibaba.
Yes.
Huh?
You know what means?
No,
brother.
It's like brother. Yeah.
This is like a central place for a lot
of factories to come buy stuff.
So like you know all the vendor
suppliers are here. So they just come
here and grab this from this guy, grab
this from this other store.
Factory usually.
repair shops. Show the
SMT.
Yeah. Everything is sold on on
Oh,
So as long as they make money, they will
like it. If they don't make money, then
look for something else.
So he's saying it's like Chinese are
different from
Chinese is like yeah it's more
practical.
Vincent's cousin connected a missing
link for us. People like him selling
FETSS, inductors, capacitors, resistors,
MLCC's, and so on for repair shops can
help to supply the parts needed for
upfit or modifications for GPUs. Someone
doing this kind of work but specializing
in video cards might source additional
VRAM, for example. He told us about how
5090s are easy to get because the local
factories supply them, matching Mr. 5's
comments earlier. Some of the sellers
here told us that they're capable of
sourcing devices like A800s or even
A100s. They just didn't have them on
site. They have them in a more secure
spot. That's because there's a separate
area for the warehousing. And
fortunately, we learned where they are
from our new friends in SEG. Like every
other link in this chain, we're getting
closer to understanding the full story
with each person who pours some tea for
us.
Yeah. Okay.
He likes chatting or hanging out.
He's like they enjoy chatting with like
different types of people like just to
like you know learn from different
culture and they enjoy they really enjoy
that. So
we agree but it was time to head back
because tomorrow
we had a morning to kill.
We wanted to find those warehouses to
figure out how many GPUs there are. This
seems like the right area. Can like
breathe hot air on it and it'll help
equalize it.
We only had a couple hours before our
highspeed rail appointment.
SCG's guards have never been
particularly friendly towards filming to
begin with, but ultimately they're not
police. The worst that happens is we get
asked to leave.
Besides, there weren't any signs saying
we couldn't be there. I I mean, I I
can't read Chinese, but I assume there
weren't.
Great. Elevator looks promising
for the warehouses. Yeah.
I'm a colorful um
distributor.
What you were thinking? I was thinking
that was like
you know racks and racks.
I was imagining that.
I think we got our Jeep used to the left
and these cartons. If they don't have
it, they come here and they buy it from
these warehouses which are on the upper
floors in the building. They just have a
carton. There's like probably a dozen
tall and about the same deep cartons
where each one should have five or six
cards in it. And then all those feed
into psycho and they get distributed to
Alibaba sellers like Vincent and then
get sent out from there. So it's uh
it's a lot of GPUs. Not all of them are
banned, but I did see a couple 5090s. So
motherboards motherboard. Yeah. I just
didn't expect this. It was It was this
like kind of secretive.
Yeah.
A little bit more mysterious.
No lights.
No.
The perspective is amusing. From the
perspective of the United States. This
floor is filled with highly illegal
items. They didn't have export licenses.
From the perspective of people working
here, this is just their job. It's not
shady. It's not some subterranean refuge
of firearms. It's just a poorly lit
warehouse like any number of other
warehouses around the world. But the
perspective of the American government
here would be different from people
working here. They just want to sell
these things and go home. On the other
hand, from the perspective of the people
working here, we are the shady ones.
Walking around filming and specifically
trying to not draw attention is not
normal. We're just noticing how nobody
cares that we're here. This is no nobody
gives a [\h__\h]
And that is called foreshadowing.
It's all individual companies. All these
all these doors are different
warehouses.
Should we go up to four or six?
Six. See what we got.
the feebleness electricity. Well,
there's a couple GPU cartons in there,
too.
It was around here that we were asked to
stop recording, so we didn't capture
what they said, but it was nima, what
are you doing? And smooth talked our way
out of it. We gave our buhisas and we
called it the successful shoot. What do
they say?
They're like, you can't record in here.
It's It's not like you have to get
permission to record here. So, we're
like, we're just documenting what we're
seeing
just for fun.
Travel vlog.
Yeah, travel vlog.
Travel vlog. Yeah.
Wait,
Raymond, we um we got kicked out.
Yes, we did.
You did your best to smooth talk them.
Yeah, we just told them that we're just
documenting to vlog and and walking
around. Yeah, they didn't really care.
Like they just kind of wanted to make
sure that, you know, nothing crazy was
going on
cuz they didn't they didn't like escort
us out. They just walked away.
No, they were like, "Oh, yeah, okay. No
worries." And then they walked away.
Yeah.
Yeah. They were nice about
Yeah.
They knew we weren't really doing
anything. We've learned a lot about GPUs
and now we're back out into Shenzhen
outside of Saig Market, a couple other
places. So, next up, we're going to go
to uh Junjo and meet with a repair shop
who builds custom 48 GB GPUs. So, we'll
do that next. Up next, Junjo.
We regrouped with Raymond and left
Shenzhen and we headed to the biggest
train station any of us had ever been
in, Shenzhen Bay, which is a high-speed
rail station. For the journey ahead, we
had a 5-hour 50minute extremely
comfortable high-speed rail ride up to
Jono.
The train goes several hundred km per
hour, which is good because we had to
travel somewhere around 1,700 km to get
to where we were going. This ended up
taking about the same time as a flight
would have taken with all the airports
factored in. But the reason we were
doing this day trip was to visit Jung's
repair shop or Brother John's repair
shop. He's famous for repair on Billy
Billy. His independent repair shop has
some Rossmanlike vibes in some ways.
He's got video cards all over the place,
spare parts everywhere. He saves
everything that he can salvage and then
rips the good components off of the
boards to reuse them for other ones. But
John is an interesting guy in this chain
because he's not directly part of the
GPU black market. What he does is fix
video cards. He does that for consumers
and for companies. Sometimes that means
people want him to modify video cards to
double the VRAM, for example. That's
where it starts getting into the
territory that's interesting for our
movie today. So, we're in Janjo. It's uh
about 6:00 a.m. Scorching hot. It's
about 30° C already and just got
breakfast. Now we're going to see our
repair shop guy to check out the 48 GB
GPUs and see what he knows about the AI
GPUs and how they get in. It's real
[\h__\h] hot.
We're just walking in the road to get
the car. This is how you do it here.
Or at least this is how Raymond does it
here. I don't know if that's how you're
supposed to do it. Okay. All right. So,
we are at John's repair shop. He's up
there right now working on GPUs. You can
see him in the blue vest. We got a pan
up there. So, we're going to go in and
work with them on uh counterfeit GPUs
and 48 gig 4090s.
just
man.
Okay.
Okay. Okay. Okay.
Okay.
Need help with that. Okay. So, uh
fe
Okay. So, there's in in a way there's
like there's some 5090s, but then you
know the because there's not that many
then you know there's not that many to
repair.
Okay. Got it. Cool. Cool.
Okay.
So, the older cards.
Yeah. This like past warranty.
Pass warranty AMD cards. Yeah.
Okay.
Okay. So even if like uh there's there's
not many like the Intel cards and then
if they want to repair it,
they're unable to repair because they
don't have the information or experience
in dealing with that.
Okay. So there's not really any info for
the Intel ones.
Yeah.
These are all finished repair ready to
go.
Yeah.
So they have all this here and they're
going to weigh it
and then have everything packed and
ready to go.
Cool. Are most of the cards that come in
local to Janto or are they from like all
over China?
Okay. Yeah. Through like his channel
through Billy Billy. Billy Billy.
Cool. Very cool.
Okay.
Oh, this is cool. This is a schematic, I
guess.
Oh, really? Yeah. It's like short
circuit or it's like missing circuit.
Are these takes a while?
Is it hard to find the schematics, the
drawings, I guess you could say. But
for older models, like for the newest
ones and it's usually, you know, takes a
while to
C2 C2. Is that what it is?
Yeah. Okay. Do they ever do GPU swap?
GPU.
Okay.
Really? That's the easy part.
Oh, really? Okay.
It's a lot easier than doing this. Okay.
So, this is actually one of their main
fixes.
Swapping.
Swapping the GPU. That's what
after after a long time usage,
right? Like, so that's
one of the things that they have to
Very cool.
We spent another 2 hours touring Junga's
shop. It's pretty amazing. We actually
have a full separate video that we're
going to put together and publish and
it's going to take me forever to
translate it. And we'll do that because
we hit our stretch goals on the
fundraiser for this movie where we said
we'll post a full separate breakout
video just of his shop because it's so
damn cool. But the real reason we're
here for this specific topic today is to
see how these cards are modified.
Because if there's an export ban on GPUs
going to China, then it becomes critical
for China to be more self-sufficient on
keeping those GPUs that they do manage
to get in service. Most of the repairs
are more typical things you'd expect of
soldering. There's board heaters,
soldering irons, surface mount
components gathered from suppliers
nearby, similar to the one that we saw
in Shenzhen, and a lot of test stations.
But one thing that's unique to China and
especially to this shop is the ability
to take an existing model video card and
completely modify the skew into
something that Nvidia doesn't even make.
Nvidia is intentionally restrictive with
how much RAM it puts on cards. Part of
that is to upsell people to more
expensive model. Actually, that's pretty
much all of it. That's basically why
they do it. But at shops like this one,
they double the VRAMm on some of the
cards that come through. It's a fully
custom skew. So, we wanted to see if he
had one lying around that he could show
us. And he did one better than that.
She can make one right now.
Okay.
Did you want me to make one?
If she feels like it, if it's not too
inconvenient.
He could do he he could do the from 24
to 48 gig
right now.
Yeah.
So
I think we're going to make a 48 GB 4090
right now. I think that's what's
happening.
GB.
This is a 24 GB 4090 that's been through
a lot of repair steps. So, if we zoom in
up here, they track the repair history
on the card in order. So, this is 1 2 3
4 5 six repair steps. The board's been
through a lot. It originally didn't work
at all. And then they had a memory
problem. They had some other problems in
between here. Uh and now it's actually
working. I think that's what that okay
is. And so it's functioning now. And you
can see it even turns on. So it's a
4090. Uh and it's got all 24 GB
detected. But because this board has
been through so much uh you call it like
trauma or something from repairs. uh
it's going to be better to harvest the
expensive components and put them on a
new board so they don't have to worry
about if it's just going to fail again
in the future from something else. So I
think the next step is maybe going to be
to put the extra memory on it to go to
48 GB. So okay
going to run a quick test
Yeah Mark
What's like the success rate of the swap
for him?
Oh, okay. Wow.
Okay. Yeah, there's still like a 1%
chance.
That's really good, though. I mean,
yeah, that's impressive.
Cool.
Oh, cool. Okay.
Okay. Protect it.
Oh, great.
To protect from static.
Oh, ESD. Okay. So, it's protecting it
from static electricity with the cover
on the PCIe slot.
And now we're going to go into the lab.
So this is the repair lab with stacks
upon stacks of GPUs. They test about 50
a day here. Because Nvidia doesn't make
a 48 gig 4090, the repair shop has to
source its own PCB. It needs something
that has enough pads for all the memory
modules they're adding to it. They work
with a thirdparty supplier that builds a
PCB, uses an SMT line to place all the
VRM components, and has the extra
wiring, circuitry, and pads to support
48 GB of memory. That's what you're
looking at here. It's a custom board
just for this. Cool.
So, I think they're going to disassemble
it now. They're just showing us the PCB.
Uh, it's made custom. He said you just
buy it from shops. They all have it. And
it's he was saying they're very common
here.
So first is disassembly.
He's going to take apart the old card.
So what did he say? Sometimes people buy
like 10 or 20 at a time or something.
Uh you get a 10 or 20 orders at a time.
And then that was like um
like numbers or model.
Yeah. for AI.
Yeah.
Running like range of running models or
right
whatever they're doing.
Does he think does Nvidia care? Does he
think Nvidia cares that people cuz this
isn't an official skew? Not that he
should give, you know, he has no reason
to care, but
Yeah.
Uh
They can't do anything anyway.
And it's after market, I guess. You can
It's kind of like souping up your car,
right?
Sometimes Nvidia blocks stuff.
Okay. like they'll do something in the
the firmware of the BIOS to specifically
try and like block a modification.
How long does this step normally take?
So, I'm going to heat it up right now to
260°C. This is really similar to what we
saw with um EVGA.
And
uh just blasting the GPU with heat.
Takes about 5 minutes and then he'll be
able to pull the steaming GPU off.
All right. So now he's pulling the GPU.
It's going to happen really fast.
He puts it on just a metal stand.
Soy
memory. So now they're using a board
heater. You can see that there's a
temperature reading on the front. uh
currently at 133 Celsius.
He's going to heat the board and then
he's gonna use the hot air station which
uh is up and right underneath a benchtop
power supply. So that hot air station
will be used to heat pinpointed memory
modules and remove them without risking
damage to other components on the board.
And so this is what Lewis Rossman uh was
talking about in our video about Teriffs
previously. the thing he sells similar.
So they keep the original memory
modules. So he's going to pull these and
keep them. And then they also have new
modules they bought sounds like just
from TAO basically is retailer here. And
they're going to add those to the board
so they can they can double the memory
capacity.
Cool.
I'm quiet. Quiet.
Wow.
So, the board heater is at 320°
and the hot air station's at 396°
C.
This part is done. Okay. So, right now
they have this
uh template for the solder balls. So,
this guy is currently he's cleaning it
looks like with mostly rubbing alcohol.
That's a cool dispenser. Uh so, he's got
wipes it down. He's going to pour solder
balls into the uh the template and then
use that to position it on top of the
memory modules. They do eight at a time
and that'll allow them to bake the new
solder balls onto the memory modules
which they can then attach to the PCB.
So you can see it's actually sized for
the memory modules.
Looks like now he's uh he's already
applied flux and he's
just wetting the solder balls.
Okay. So, now he's using a wick. So,
this is a solder wick. Uh, it's just a
piece of copper that basically pulls off
the excess solder. So, he's running that
over the surface.
What do uh 48 GB cards normally cost?
It's like what do they normally sell
for?
20,000
20,000 plus
2,800 US.
That's not bad. Actually,
the soldering ball.
Yeah.
Okay. So, it's kind of like a what we
call it? It's kind of
have everything kind of filter through,
right? Uh,
okay.
Okay. So, they they're going to add the
heat now, right? That's what he said.
Okay.
Oh, okay.
Cuz he needs to do another one, right?
Yeah, he's going to do another one cuz
he could only do eight
at a time.
At a time basis.
Okay.
Cool.
Yeah.
Now he's adding heat. So, the hot air
station is currently running at 390
plus or minus degrees Celsius.
He's directly applying heat to the
surface to where the solder balls are.
And then he's got the memory modules
themselves heated from underneath by a
hot plate. So, the hot plates plugged
in. That's running at 195° C. And now
he's just holding it down while he
presses it into the hot plate.
Is it hard to get uh like 4090s or 5090s
for him?
customers bring it. He doesn't have to
buy he doesn't actually buy just repair.
Got it.
It's just so crazy to me how like
there's a couple shops in the US that
that can do repairs,
but they don't really do like this level
of repair. They
Okay. But I I don't know of a single
shop in the US that does like the memory
modding.
Yeah. Cuz labor cost is a huge issue,
right? For the US. For him, it's like
it's worth it. Uh for the customers,
it's worth it.
Yeah.
Yeah. Cuz it doesn't
like this the labor cost or the service
cost is not as high.
Oh,
two hands.
Oh, okay. Okay. Okay.
So, next stage at this point, they've um
reballed all of the memory and the GPU.
And
now he's cleaning the surface of it to
prep it to mount to a new PCB. We saw
the new PCB earlier where it's it's a
brand new custom PCB they just ordered
from Tao, which is the retailer. It's
adding flux now.
Lewis Rossman says you can never have
enough flux.
And so that's going to help it
when he melts the solder to apply it to
the board. Another custom. So there's a
custom template
just like for the memory. He's going to
apply that to the surface.
And now he's got a tray to catch the
excess.
And he's gonna just Yeah. basically like
sifting for gold or something and sand.
So he's sifting it.
Wipe off the extra.
And then carefully remove the template
while leaving them all in place. And
they should be held in place by the
flux.
So, this is a jig to hold the GPU in
place. I guess the question is, is that
to protect the GPU from the heat?
I guess. Are they putting the
stuff on the card now?
This is a new one.
Okay, so now we are the GPU is on there.
So we're at the final stage. This whole
process is like about 2 hours, I guess.
And now we're doing the
the opposite of what happened earlier in
the day.
So now instead of removing the GPU, he's
adding the GPU. So he's very carefully
positioning the GPU
to precisely align the PGA with the
solder balls.
And now he's left the room. So, I guess
Tannon's going to take over from here.
Interesting. So, it looks like he's
starting to reach steady state on the
temperature. A couple hundred degrees.
Ton of
ton of bubbling going on at the edges,
but uh they're just baking the GPU back
on. So, at this point in the process,
the PCB has been swapped. The PCB
already had all the VRM components and
the IO populated 12vt high power. And
now they've added new memory to the
front and the back. One's got reused
memory, one's got brand new, and then
the GPU has been pulled and swapped and
is now being remounted. And they're
telling us this is a 99% chance success.
This is what Kingpin called the nervous
part.
Okay, so he's going to cool it off with
the fan.
Do they reuse the same cooler that they
took off the old board or do they need a
new cooler now?
Dude,
that's the
cool snapping.
Moment of truth.
Moment of truth. Yeah.
So, he's just going to hold the heat
sink on and see if it powers on.
Cool. Okay. So, now he's testing to see
if it actually powers on after all that.
It's got a green LED with displays up
debug up top.
And he's just holding a heat sink in
place cuz all he needs is power to see
if it spits out display.
We got display up top.
So it is functional. Now it's just a
question if everything detects.
I know.
We're lucky. He's like usually it
doesn't turn on the first time in a
while.
They know we have a train to catch.
Yeah.
Does he know? Do the customers who buy
these, do they mostly put them in
servers?
Yeah. Like if the customer buys more
like multiple units, then it's probably
like, you know,
server. Yeah.
Yeah. That's why the blower fan would be
better for servers cuz they pack them in
so tight.
I was just asking if they have different
choices for the cooler
and then they're like there's only one
they only they don't have a choice
basically.
Okay.
So, next step, one of the last steps in
the process since it booted and worked.
Now, they are pulling the protective
film from the thermal pads. There's a
PTM or actually is that that might be a
paste actually pre-applied.
Looks like it was just silk screened on
there.
Looks like it works. So within 2 hours,
start to finish, we have a working 4090
48 GB. At the beginning of this process,
the card was working, but it had been
through so many repairs that it was very
likely it'd break again at some point
from something they couldn't really
predict. First of all, from a waste
standpoint, it's just way better to
figure out a way to save that card
because the PCB is kind of the least
valuable thing in all of that and is
also the least precious of the
resources. being able to save the
silicon that's in the memory and in the
actual GPU itself that has a lot of
value but especially has value in China
where they just don't have as much
supply. Even though the supply looks
pretty damn good from what we saw, it's
still intentionally restricted. So
purely from the perspective of the
people living there, not only is this
better than what Nvidia shipped to begin
with, and really not that far off in
price when it was at its most scalped,
it's also just a good way to kind of
keep the silicon in circulation even as
things like MOSFETs, capacitors, or PCBs
die. The silicon tends to be pretty
resilient and it doesn't really die that
often on GPUs. So, what this tells us is
that this is maybe an alternative method
to getting more GPUs into supply. If
they take the broken boards and then put
functional silicon back into
circulation, it's a certain level of
ingenuity, and there aren't many places
that are set up to handle it. Brother
John's shop is impressive. We're excited
for the whole separate video on that.
After finishing at the repair shop, we
rushed over to the highspeed rail. Spent
six hours traveling back to Shenzhen
Bay, adding another thousand miles or so
to our travel log here. And then we took
a car over to visit this guy. He calls
himself the Chinese M&M.
Eminem and China.
So, this is Joe. He works as a creative
director of product at the video card
manufacturer Yestin. Most famous for
their uh waifu video cards. You know the
ones.
Yeah. Yeah. I'm he his father.
She's father.
Yes. She's father.
Yeah, we have husband.
Anything else you want to talk about on
camera? I think we got it all.
Game natur.
Yeah, I also have a same nickname.
Which one?
Yes.
of
what does that mean in English?
Uh, yes. Uh, yes. Tough guy.
Tough guy.
All for I will put your head
on the ground. Now,
Joe and Yes were not officially
contributing to the black market side of
the research here, just to be clear to
their own partners. Instead, they showed
us around their factory. This will be a
separate upload later with an actual
factory tour with them showing how the
waifuss are made. But in this particular
video, there were a few interesting
things we learned that we thought fit
here pretty well. The big takeaway from
the factories is that they receive their
GPU supply, of course, from their
partners. That would be AMD, Intel, or
Nvidia, and the memory is often packaged
with it. But for the most part,
everything else that they use, unless
they're buying reference PCBs, all that
comes from whatever sources they want to
find for their supply. We were also
informed that there's very little
oversight in terms of the management of
the rejects at these places. They can
file for refunds if, for example, a chip
is bad. But if they screw up a board in
their own process, like the SMT lines
miscalibrated and optical inspection
doesn't catch the error and they place a
MOSFET on backwards and blow something
up or whatever, then it's going to be on
them to deal with the defect. Now, this
type of thing basically never happens.
All of this stuff is highly automated.
The point of pointing it out though is
that for at least the gaming class
cards, which the 5090 more or less is
one of, there's enough volume and
Nvidia's hands off enough that it would
be pretty easy to make these disappear.
The biggest reason for this is that
there are partners for the 5090s. It's
not like a card that only Nvidia makes,
such as the RTX Pro 6000 and the
Blackwell edition. And since there are
partners, Nvidia is only tracking when
they sell them the GPUs. they don't
necessarily keep tabs on where those get
distributed once they're done being
manufactured. So, for this reason, if a
factory wants to just sell it
domestically in China and they think
they can get away with it without being
added to the entity list, which it's
very likely would kill their business
with Nvidia, then they can go ahead and
sell them domestically. That might be
how some of these trading companies get
a hold of them. It's an easier way to
make them disappear than through
channels that are more publicly visible
from Nvidia and government. Now, Yestin
doesn't make 5090s, but the processes
they taught us about illustrate how that
could happen. As for the server grade
solutions, there are manufacturers for
those that are pretty tightly
controlled. Basically, the silicon is
made in Taiwan. It's shipped to wherever
it's going to be assembled. Almost all
of the other components are sourced in
China. And as a result, it's easiest to
keep the whole supply chain there,
including the assembly. And the assembly
involves the same stuff you're seeing
for the lower-end cards and the AMD
solutions we're looking at from in this
factory line. They use surface mount
technology or SMT lines with conveyed
inflow of PCBs, reels upon reels of
components that are placed smallest to
largest with the most valuable typically
at the very end. That'd be the GPU. And
then manual placement for heat sinks.
There is often manual assembly even for
Nvidia's high-end stuff. Although we
can't show you that here, we have seen
it being made. That's in different
factories. And at the end of the
process, it's possible they find a
defect. Again, Dr. Vinci Chow had one
where he has a broken link on his board.
There's a good chance that stayed in
China and never left. If it was reported
as a defect to Nvidia, then it may have
effectively just been written off and
considered trash. But after seeing how a
highly skilled group like Brother John's
Repair Shop a second ago can just
repurpose all those components that are
still good, you could see how there
would be value in figuring out a way to
make that board disappear, even if it's
supposed to be in the trash. This comes
back to Nvidia's turning a blind eye
because they'd eventually notice it from
the serial number popping up somewhere,
but maybe they choose not to. And
besides, once someone has it, especially
in China, there's not a ton they can do
anyway. That's a look inside the
manufacturing process, which we felt was
an important piece to this story. Let's
move on to the next stop. My favorite
place to go in the world, Taipei,
Taiwan.
Most of our stay in Taiwan was for other
stories. However, we did accidentally
unturn one stone with something
interesting under it. We can't show any
B-roll for this because we do protect
sources, so we're just showing some
other footage instead. A B2B company in
Taiwan noted to us that it's
commissioned by Chinese companies to
import servers for pre-esting and
pre-assembly and setup steps. When we
asked what the GPUs were that the
servers ran, having just done all the
rest of this, we realized that all of
the hardware is export controlled. Once
the B2B agency completes its testing, it
ships the system out to the original
buyer. Basically, they act as an
intermediary to buy the machine, mark it
up, and reship it to the original
purchaser, maybe with actual testing. We
also spoke with a company that conducts
business in Singapore. The company
informed us that they are also aware of
similar passroughs for so-called testing
or data services. Our last link in this
chain is actually the first, the
smugglers, bring it full circle. And the
smugglers themselves also put us back in
the United States. We didn't think we'd
be able to find someone doing the dirty
work here, but in the final hours, a
viewer had a lead. This led to about a
12 or 16 hour production delay as we
were wrapping the project, but as far as
we're aware, this is the first content
piece that actually features someone
doing the highest risk dirty work.
Everyone else we spoke to is in China.
They're safe from US retaliation. But
large-scale smugglers get arrested, find
millions of dollars, and can spend years
in prison if they were really serious. A
viewer contacted us to say that he'd
connected with a traveling GPU buyer.
The story went that this guy drives
around buying specifically RTX490s. He
doesn't care about anything else,
including 5090s because those don't sell
as well to China. 4090s are ideal, he
told us, because they can be modified
into cost-effective 48 GB models like we
saw at Brother John's shop. The smuggler
was extremely open with us in text
messages written only in Chinese, but
wasn't open to us flying out to him. He
did allow us to share this video of his
car, though. The plug, as we call him,
has an ATX test bench, motherboard,
simple downdraft cooler, and a gigantic
battery in the trunk of his Prius. He
also has at least one spare license
plate in the trunk in addition to the
one on the back of his car. We're not
sure why, but better not to ask. People
like The Plug post on Facebook
Marketplace and other common online
reseller forums looking to buy GPUs,
just like anyone else in your city
would. The Plug offers $2,000 flat per
4090. Doesn't matter which model it is.
It's not a bad price. He told us that he
then finds ways to get them into China.
He recently was scammed out of $5,500
though of payments owed by a Hong Kong
buyer and that buyer resells in Hong
Kong. He's been considering hand
carrying the devices and luggage
instead. Margin is slim. He makes just
under $300 US per GPU profit, but that's
before counting gas, potential hotels as
he travels, and time. Our understanding
is that some in his shoes will strip the
GPU cooler off and ship just the PCB
back to China, increasing their margin.
Coolers are available in abundance where
they're made, so it can be cheaper to do
this than pay for the weight and the
size. He sometimes buys entire computers
just to take the 4090s out as that's
what the bounty's on. He then sells the
remaining system back to whoever will
buy it. He told us that 5090 prices in
China are falling so fast due to over
supply, ironically, that he'd lose money
reselling it to China and would do
better flipping it to Americans. The
plug isn't rich. She seems to be doing
okay, but it's not the type of wealth
you might expect for taking such risk.
We can't overstate though how important
it was to get this piece of the story as
this allowed us to fully complete the
chain. And a huge thanks to our
anonymous viewer who set that up. Many
of the sources we met in Asia during
this trip told us they simply don't know
how the GPUs actually get in. And now we
know people doing this on a small scale,
like the plug, are unlikely to be
caught. But operations transacting
millions of dollars worth of GPUs would
have a harder time getting them out
undetected. And that brings us to the
end of our travels, but not the end of
this piece. We have two more entities to
look at and that's Nvidia and the US
government. The legality is simple. This
is our understanding of it. There's no
restriction on purchasers only on
sellers the US has control over. Even
Americans buying GPUs in China are not
violating any laws. It is not
restricted. So, we'll start with
selling. The simplest answer to who the
sale of GPUs is restricted for is
anybody who doesn't have a specific
export license and who would be governed
by US law and Nvidia can't just bypass
it by shipping from a different country
instead. It has to do with the sale, not
with the shipment location. That
includes entities buying from US
companies like European companies who
would be under guidance from both Nvidia
and if they do business with the US, its
government. It's also illegal for
anybody in the United States, citizen or
not, to sell these GPUs to China, Hong
Kong, Macau, or companies in those
locations if the seller does not have a
reexport license. It is not illegal for
a person of any nationality in America
to sell a GPU to any non-restricted
entity. As for buying,
there's absolutely nothing illegal about
for us over here to buy these GPUs,
but it's also not illegal to sell them
in China for a Chinese company. The
Chinese government doesn't generally
enforce American export laws. Other
nations cooperating with the United
States might, such as recent arrests in
Singapore. But once the GPU is in China,
the people in possession of it likely
don't care. Buyer or seller, or someone
who does both, like GPU dealer Vincent.
The only control over Chinese companies
that the US has is the entity list,
which would hurt their business
prospects with American companies, but
only if they care about that. Chinese
GPU middleman Sutan Halei in Tianin
landed on the entity list for
transacting banned GPUs and being found
out. Sutang Halei purchases GPUs from
anyone who can get them into China and
then they bid on domestic projects like
data center buildouts. The entity list
was used to restrict for example
Deepcool previously resulting in their
American partners ceasing business with
them for fear of frozen assets, audits
or collateral bans. So this hurts
companies like Deepcool that want to
operate in the US and it even shut down
their California office. But for Sutan
Huli, they mostly want to do business
with other Chinese companies and so it
have limited impact unless they wanted
to expand to do business with Americans.
Beyond the entity list, the US really
has no control over what happens inside
Chinese borders. That means the only
point at which a GPU could feasibly be
intercepted and a person arrested would
be operating in the US or in transit to
intermediary countries such as
Singapore, which may have their own
export laws. The people buying and
selling them within China are not
breaking any of their own government's
laws, though. And so, we come back to
Nvidia. At every turn, it really looks
like Nvidia is playing all sides. If
there's enough money to be made, anybody
is Nvidia's friend. We think the story
of AI GPUs has become a story of
corruption between governments and the
wealthiest company on Earth. For
example, on April 30th, Amazonbacked AI
startup Anthropic called on the US
government to increase export control
restrictions to China. As part of this
blog post, Anthropic said the government
needs to improve its export enforcement
to reduce smuggling. The company cited
examples of chips being smuggled with
quote prosthetic baby bumps end quote
and quote live lobsters end quote. This
upset Nvidia obviously because Nvidia
doesn't like restrictions to making
money. So Nvidia shot back, seemingly
taking a page out of Trump's playbook.
Nvidia essentially called this fake
news. Quote, "American firms should
focus on innovation and rise to the
challenge rather than tell tall tales
that large, heavy, and sensitive
electronics are somehow smuggled in
scare quotes bumps or scare quotes
alongside live lobsters end quote."
Except Anthropic isn't telling tall
tales. It's right. A 2022 video that we
uncovered showed a security check at
Duhai, a port in Guangong. It's not far
from Shenzhen, wherein a woman with a
prosthetic baby bump was shown to have
been carrying CPUs and iPhones instead.
The report made it to customs.gov.cn,
stating that she arrived from Macau, a
common gobetween, similar to Hong Kong.
This story got international attention
in technical media, and we reject the
possibility that Nvidia wasn't aware of
it. We found the official Chinese
government posting about smuggling from
Macau. Imports to most of China are
taxed and so tax evasion coupled with
smuggling will increase margin on the
electronics rather than sharing it with
one of the two governments. If you're
already breaking a US law, it seems some
just go for a hat-tick and increase the
profits. As for the lobsters, that's
real, too. We unearthed this official
Hong Kong customs website detailing a
white van busted driving on the Juhai
Macau bridge filled with 280 kg of
undeclared live lobsters and yes 70
smuggled GPUs complete with photo
evidence. The Hong Kong customs itself
calls this so or smuggling and notes a
maximum sentence of 7 years. Nvidia's
response to call these tall tales then
is not only defensive but serves to
gaslight and grossly mislead we think
and is tantamount to lying for sake of
downplaying reality for its own benefit.
But then this is a common Nvidia tactic
including its dishonest approach to
reviews that we've already detailed in
our opinions and its deceptive and we
think false advertising of the RTX 5070
as being equivalent to an RTX 4090 which
is provably and comically false. Another
instance of Nvidia's fake news defense
was following a July 24th publication by
the Financial Times, reporting that more
than a billion dollars of Nvidia AI
chips had been smuggled into China. In
response, Nvidia, whose blind eye is
turning an awful lot lately, downplayed
the issue, stating, quote, "Trying to
cobble together data centers from
smuggled products is a losing
proposition, both technically and
economically. Data centers require
service and support, which we provide
only to authorized Nvidia products." End
quote. The last part sounds like
something a company selling support
would say. And the first part doesn't
really match Nvidia's whole the more you
buy the more you save thing. It's only a
winning proposition by all of their
prior years of statements. If your only
option is a useless insufficient data
center or a cobbled together sufficient
data center, then a cobbled together one
is still a winning proposition by
comparison. It's weird for them to
pretend that this isn't worth doing.
It's worth like lots of money. That same
day, white house.gov gov posted an
article titled why a to claim for
President Trump's visionary AI action
plan. Near the top of the post, it
highlighted a sickopantic quote from
Jensen Juan which read quote America's
unique advantage that no country could
possibly have is President Trump. End
quote. And we would call that
sickopantic if it's about any president.
On August 5th, Nvidia got another
opportunity to talk. The US Department
of Justice announced that it had
arrested two people in California for
smuggling quote tens of millions of
dollars worth of sensitive microchips
used in artificial intelligence
applications and quote to China. The
court documents say the shipments
included the Nvidia H100 and the Nvidia
RTX490. Rather than admit smuggling
exists, Nvidia downplayed the situation
again and stated this time, quote, "This
case demonstrates that smuggling is a
non-starter." End quote. except that
anyone who made tens of millions of
dollars before getting caught had a
pretty good start and so did their
customers. This is a [\h__\h] statement
from Nvidia that seemingly aims to
downplay and deflect to reduce lawmaker
attention on its monopoly. Nvidia also
said, quote, "We primarily sell products
to well-known partners, including OEMs,
who help us ensure that all sales comply
with the US export control rules."
Nvidia noted that quote, "Any diverted
products would have no service support
or updates." End quote. Again, this is
not fully true. Our own sources in this
story noted that although something like
an HDX system would be hard to service,
a standalone PCIe GPU could be parted
out and covered under a separate
warranty, even in China. Meanwhile, the
US began scrutinizing Nvidia's
technology for getting into China,
whether or not the company itself was
directly involved. In a bipartisan
report called Deepseek Unmasked, a
government committee called Deepseek a
quote threat to national security end
quote and said Deepseek had used
Nvidia's technology. quote, "AI model
appears to be powered by advanced chips
provided by American semiconductor giant
Nvidia and reportedly utilizes tens of
thousands of chips that are currently
restricted from export to the People's
Republic of China or PRC. NVIDIA
designed and manufactured many of these
chips to create the most sophisticated
possible chip while skirting US export
controls. This has allowed these chips
to be exported to China as the US
government develops stricter
restrictions. Since March 2024, it is
estimated that Nvidia has produced over
1 million chips for the Chinese market.
End quote. The government also examined
Nvidia's significant revenue growth in
Singapore compared to China based on SEC
filings, particularly in years featuring
restrictions. The government questioned,
quote, whether PRC customers are
arranging for the diversion of sensitive
chips that are reportedly sold through
Singapore, end quote. This was stated
because revenue from Singapore had grown
from almost nothing since 2020 or 2021.
Nvidia has defended its sales to
Singapore by stating that quote
customers use Singapore to centralize
invoicing while our products are almost
always shipped elsewhere. End quote.
According to Nvidia, shipments destined
to Singapore were only 2% of the
company's total revenue in 2025. But we
also know that Singapore has made
numerous arrests relating to GPU
smuggling. So, there appears to be some
reason for the concerns whether or not
Nvidia itself wants to turn a blind eye
to it. Despite the allegations, Nvidia
downplayed any smuggling of AI chips. In
this video uploaded two months ago from
what appears to be either a potato or a
VHS tape, Jensen Juan speaks on
smuggling.
Governments understand that diversion is
not allowed. And there's no evidence of
any AI uh chip diversion,
except that there is evidence of it, not
only in this very movie, but in readily
available reports online for years now.
Huang continued,
"Our data center GPUs are massive. These
are massive systems. The Grace Blackwell
system is nearly two tons. And so you're
not going to be uh sh you're not going
to be putting that in your pocket or
your backpack anytime soon. And so these
systems are fairly easy to keep track
of. And um and but the important thing
is that the countries and the companies
that we sell to recognize that diversion
is not allowed and everybody would like
to continue to buy NVIDIA technology.
And so, uh, they're they're very well,
uh, they monitor themselves very
carefully and and, uh, they're quite
careful about that.
This one's interesting. He's right in
that it is much harder to smuggle Grace
Blackwell or Hopper HGX class systems
completely. Dr. Vinci Child's statements
align with this.
It's very hard to get like a full HGX
system, right?
But it still happens. At least one of
Nvidia's GPU and server customers in
another country told us that they
facilitate intermediary transmission to
China and in fact showed us the server
racks on site in their facilities. We
weren't allowed to film them, so this
footage is from Computex instead, but we
saw them. A separate representative told
us that document forgery through third
party countries can also disguise such
trans shipments. One middleman told us
on camera earlier that an Nvidia
distributor gets parts into China, a
downchain factory, and not the one we're
showing right now, in case Nvidia has
its famed journal of retaliation out,
told us that Nvidia's QC rejects
sometimes end up repurposed and kept in
China, salvaging the GPU and the VRAM
and scrapping the rest. Dr. Vinci Chow
told us that one of his own devices had
a defective link on it, contributing to
this statement because he acquired said
device. It just happens to have a
defect. And he also said this.
Do you think Nvidia
knows that all of this is happening?
I would be surprised if they don't.
Right.
Yeah.
I would be surprised. I would be really
surprised if they don't.
I see.
Right. These are very expensive items.
Yeah.
Right. I would imagine you would keep
track of everything. Right.
Yeah.
I'm not I don't know. Is that
I'm don't Yes. It's hard to know how
what they going to do, how what what
they plan to do with all these uh
defective parts.
But I'll be very surprised that no one
have ever thought of the possibility
that if it's something so valuable,
someone would come up with a use of
even a defective one.
Defective ones.
Yeah.
So in one set of statements, Nvidia says
that smuggling doesn't really happen
because the export controls work and
keep partners in line. But in another
statement, Jensen Juan called the US
export controls a quote failure end
quote talking out of both sides of his
mouth. We think he spoke of competing
Chinese GPU brands posing a threat to
Nvidia. Quote, "The local companies are
very, very talented and very determined,
and the export control gave them the
spirit, the energy, and the government
support to accelerate their development.
I think all in all, the export control
was a failure." End quote. But the
stakes and the dollar signs for Nvidia
had increased. Jensen said Nvidia's
market share in China had dropped from
95% to 50%. And in Nvidia's May
quarterly earnings before the age 20
exemption and revenue share, Jensen Juan
said the company's data center business
in China was done. Quote, however, the
$50 billion China market is effectively
closed to US industry. The H20 export
ban ended our hopper data center
business in China. End quote. Or as
Jensen says, the China market is worth
one Boeing. Uh the Chinese market in a
couple years is probably about $50
billion. Uh the the market we've left
behind is utterly gigantic. $50 billion
just so you so you have a feeling for
that number. $50 billion is like Boeing.
Probably not the best example, but okay.
Let's look back at the timeline one more
time for these export controls on
Nvidia. This time we'll focus on
highlighting Nvidia's side of things,
which is its relentless appetite for
global domination. The October 7th, 2022
Biden administration export controls had
a stated goal of quote protecting US
national security and foreign policy
interests end quote by implementing new
export controls restricting China's
ability to build high-end semiconductors
including for the development of
supercomputers. There was an included
goal of staving off the potential for
China to develop quote nuclear weapons
and other military technologies. End
quote. Blocking the H100 and A100 led to
Nvidia creating an export compliant A800
at about 70% of the speed of an A100 for
the Chinese market. A year later, on
October 17th, 2023, the US Department of
Commerce updated its export compliance
and restricted Nvidia's A800 chip as
well, alongside the newer China targeted
H800. Weeks later, on December 6th,
2023, Nvidia told reporters in Singapore
that it would be working on another new
chip that would comply with the US's new
restrictions. On July 13th, 2025, which
was after Donald Trump won the 2024
presidential election, but a week before
he took office, the outgoing Biden
administration tightened export controls
by introducing national chip caps for
many countries except for 18 allies.
That's the AI diffusion rule we spoke of
earlier, and it would have gone into
effect in May. Nvidia, who had remained
relatively quiet about the regulations
up until this point, criticized the
restriction and made an attempt to
appeal to the president-elect, stating,
quote, "It makes no sense for the Biden
White House to control everyday data
center computers and technology that's
already in gaming PCs worldwide,
disguised as an anti-China move. The
extreme country cap policy will affect
mainstream computers in countries around
the world, doing nothing to promote
national security, but rather pushing
the world to alternative technologies.
AI is mainstream computing, ubiquitous
and essential as electricity. This
lastminute Biden administration policy
would be a legacy that will be
criticized by US industry and the global
community. End quote. Seems like Nvidia
tried to also set up an appeal to the
president-elect, stating this quote, "We
would encourage President Biden to not
preempt incoming President Trump by
enacting a policy that will only harm
the US economy, set America back, and
play into the hands of US adversaries."
End quote. This is where in the timeline
we'd hit the million-dollar dinner and
then the ensuing ban unban of the H20
chip, but we've already covered that.
Later that month on April 30th, Juan
said this of Trump. Without the
president's leadership, his policies,
his support,
and very importantly his strong
encouragement,
and I mean a strong encouragement, I
frankly frankly uh manufacturing United
States uh wouldn't have accelerated to
this pace. A month after the Mara Lago
estate dinner on May 7th, the US
Department of Commerce confirmed that it
will not implement the AI diffusion rule
that Nvidia campaigned against and that
was created under the Biden
administration. The rule was supposed to
go into effect a week later on May 15th.
This does not unban GPUs like the H100,
59, AB100, and so on. Following the
termination of what was supposed to be a
rule to address national security
implications, the Department of
Commerce, now under President Trump,
stated, quote, "The Biden AI rule is
overly complex, overly bureaucratic, and
would sty American innovation. We'll be
replacing it with a much simpler rule
that unleashes American innovation and
ensures American AI dominance." End
quote. Nvidia predictably celebrated
this statement. The company, which has
been begging to sell to China while also
praising Taiwan's importance, now took
an America first posture, collecting
countries like Pokémon. Quote, "We
welcome the administration's leadership
and new direction on AI policy. With the
AI diffusion rule revoked, America will
have a once- in a generation opportunity
to lead the next industrial revolution
and create high-paying US jobs, build
new US supply infrastructure, and
alleviate the trade deficit." End quote.
Job creation promises coming from this
company in particular are interesting,
but they are playing all sides here
consistently. Later that month on May
28th, Juan spoke with Mad Money host Jim
Kramer. He stated, quote,
when he rescended the AI diffusion rule,
um it was a visionary move. It was a
bold move and he recognizes that there's
an AI race and we're not alone and he
wants America to win. June 23rd via
Reuters, an official of the US State
Department, which didn't reply to Gamers
Nexus except the department did send us
three out of office autoresponders,
warned of deepseek military and
intelligence operations and warned of
the use of shell companies, they called
them, in Southeast Asia to circumvent
export restrictions. The report
mentioned that Deepseek had quote large
volumes and quote of high-end H100 ships
which are banned in China. Nvidia didn't
like that and responded to Reuters and
stated, quote, "We do not support
parties that have violated US export
controls or are on the US entity lists,
also saying, quote, "With the current
export controls, we are effectively out
of the China data center market, which
is now served only by competitors such
as Huawei." Turning a blind eye to the
situation, Nvidia added, quote, "Our
review indicates that Deepseek used
lawfully acquired H800 products, not
H100." End quote. Three days later on
June 26th, the information reported that
Deepseek's next AI model had been
delayed due to a shortage of Nvidia AI
GPUs in China. This directly contradicts
Juan's comments that export controls,
according to him, do not work. July 4th,
Bloomberg reported that the Department
of Commerce, which also did not reply to
Gamers Nexus's emails, was preparing new
export controls on Malaysia and Thailand
to reduce chip smuggling. Interestingly,
Singapore, which now comprises a
significant portion of Nvidia's revenue,
was not on that list despite being a
known smuggling pass through. July 10th,
Bloomberg reported that Juan and Trump
were scheduled to meet again ahead of
the CEO's planned trip to China. Days
later, on July 14th, Nvidia confirmed
that it will resume sales of H20 chips
to China with Juan stating, quote, "The
US government has assured Nvidia that
licenses will be granted and Nvidia
hopes to start delivery soon." End
quote. On the 15th, a CNBC business news
anchor said this.
He really just pulled off something that
few tech CEOs have managed. He played
both Washington and Beijing and he won.
Jensen has stayed disciplined and
diplomatic with a clear message and that
is Nvidia's dominance serves America's
interest.
On the 28th, the Financial Times
reported that the US Commerce Department
was not going to make tough moves to
tighten export controls to China. This
is in spite of several congressional
members warning the administration not
to loosen the US export controls for AIG
GPUs. Several national security experts
also voiced concern by sending a letter
to the US Commerce Department which
read, quote, "We write to express our
deep concern over the recent decision to
resume exports of Nvidia's H20 chips to
China. As policy makers and
professionals with a background in
national security policy, we believe
this move represents a strategic misstep
that endangers the United States
economic and military edge in artificial
intelligence." End quote. On August
11th, we got the comments about the US
potentially allowing Blackwell chips to
be sold to China as well for 30 to 40 or
50% of a revenue share. And that brings
us to today. Nvidia might have started
off as a much more humble company. But
it has become a savvy political player
in a game that is seemingly pay to win.
That seems only fitting for a gaming
company to be particularly good at pay
to- win games. There's no one better
equipped to play that game than actually
the most valuable company by market cap
in the world. Now at $4 trillion, led by
a man whose net worth is estimated at
$148 billion. Nvidia knows when to bite
its tongue and how to effectively appeal
to the ego of politicians of all parties
and all countries. We think Nvidia is
playing all sides. We think it's greedy,
manipulative, and carefully employs
propaganda, such as its use of this fake
news playbook for news which was
literally reality. But we don't think
Nvidia has a particular set of beliefs
beyond just making more money. We think
Nvidia will sell anybody out to make a
buck. Nvidia is in the big leagues now.
It's not like the 1080 Ti. inside of one
month reportedly paying $1 million to a
sitting president after which followed
the unlock of five half billion dollars
of lost H20 revenue followed next by a
15% split of that unlock going to the US
government is what raises these new
questions of Nvidia's
integrity but we've kind of already
talked about that over the last year now
as for the actual black market side of
things and smuggling that was a cool
story this was like crazy exciting to
work on I loved actually being out in
the field and meeting all these like we
we set up some of it in advance, but a
lot of people we met on the ground
there, we were able to like actually get
to the bottom of something. It's pretty
cool. We've learned that common methods
include factory so-called QCD defects or
sometimes real ones where they repurpose
hardware that might otherwise have been
written off by especially Nvidia as just
trash, but people with the right skills
can reclaim it. We learned about the
other method of hand carrying items in
like with students or actual smugglers
on the ground in the US facilitating the
transfer of the GPUs. And of course, we
learned about suppliers through third
party countries among other methods. So,
we did answer the question that we had
going into this, which was all about how
these cards get in. Again, we loved
working on this story and meeting all of
these unique people. Each person played
a key role in helping us find the next
person and allowing us to complete the
first public on record start to finish
cataloging of a smuggling pipeline for
high-end silicon. Our sincere thanks to
everybody who made this possible and
appeared on camera with us, including
our viewers who funded it. And a big
thanks to my team working on this
project. Vitali, Andrew, and Mike spent
about a month training Tannon how to use
a camera cuz we needed a camera operator
for this trip and he's a writer. Vitali
helmed the overall production
organization of the files and cutting of
the 12 hours of footage initially.
Andrew led the artistic efforts, worked
on the new products, did the great scene
transitions you saw and general
graphics. Tannon was on the camera and
worked with me on Asia for 3 weeks on
the story and during a lot of travel.
Tim on the team handled the bulk editing
of theformational sections and the
timeline for this video. and Ben
compiled hundreds of pages of research
and worked directly with me throughout
the process to figure out what the hell
this story was going to be. A thanks to
Ryan for doing the black market logo as
well and all the other contributions in
between like from Aaron Pauly and
Matthew Kachihifi on some of the music
contributions. Thanks for watching.
Subscribe for more. We have more of
these in the works. You can go to
store.gamersac.net to support this
directly. Thank you.