One day I decided I wanted to be able to "see" what was going on inside the RAM in my linux desktop computer. And now through the magic of LLM AIs and some time, debugging, and tweaks, I can. I haven't ported it to gtk3 yet so it's gtk2 only for now.
memgaze.pl (94KB, requires libperl-gtk2 and 'aplay' binary for sound)
memgaze is a linux process virtual RAM exploration tool. It creates visualizations as images of the *virtual* RAM of any accessible process (run with sudo) and can save them as PNG. It also can play any RAM dump or subsection(s) of it as a sound file or save to wav. It has tool-tip overlays that show what type of RAM allocation it is and what, if any file it's associated with on disk (or heap or stack or anonymous) on both the hilbert map and image map.
To make the image memgaze walks through massive strings of ram dump bytes, taking them three at a time to form each pixel. Each byte has a value from 0 to 255. Coincidentally each color channel in a standard 24-bit RGB pixel is also represented by a value from 0 (no intensity) to 255 (full intensity). This allows for a 1-to-1 mapping.
The 1st byte becomes the Red component of the first pixel.
The 2nd byte becomes the Green component of the first pixel.
The 3rd byte becomes the Blue component of the first pixel.
The 4th byte becomes the Red component of the second pixel.
The 5th byte becomes the Green component of the second pixel.
And so on...
You might think this would lead to inconsistent coloring just because something shifts by one byte but it's not. The same libraries always look the same way when loaded in ram and visualized this way. Even between reboots and across different computers. And with time I'm starting to see how different types of RAM allocations have different types of textures. The data really does speak for itself.
For generating sound it's different.
But first a diversion: I've found some really interesting sounds in various programs. I really like libgtk-3.so it has some nice bass. Here's some musical tracks that I selected as truly "computer generated" music. Listen to this while you read how it works.
The stream of bytes from memory is a sequence of points that define a sound wave. Each byte's value is interpreted as the amplitude (loudness or position) of the sound wave at a single, tiny moment in time.
Unlike the image which groups bytes into threes, the sound translation treats every single byte individually. For sound the RAM dump is interpreted as unsigned 8 bit values and each byte is treated as an unsigned integer from 0 to 255. This number represents the position of the speaker cone at one instant. A value of 128 is considered the "center" or silent position. A value of 255 tells the speaker to push all the way out. A value of 0 tells the speaker to pull all the way in.
A sequence of bytes like 128, 150, 200, 255, 200, 150, 128, 100, 50, 0, 50, 100, 128... would create a simple smooth oscillation (a pure tone).
For example a snippet of memory containing the ASCII string "Hello..." followed by some binary data.
Hex Value Decimal Value Interpretation as Sound Amplitude
0x48 72 Speaker cone is pulled in from center.
0x65 101 Speaker cone is still pulled in, but closer to center.
0x6C 108 Getting even closer to center.
0x6C 108 No change from the last sample.
0x6F 111 Still pulled in.
0x00 0 Speaker cone pulls all the way in (max negative).
0xFF 255 Speaker cone pushes all the way out (max positive).
0x7F 127 Almost perfectly centered (nearly silent)
When played back at 8,000 bytes per second (8 KHz sample rate), this sequence creates a series of rapid clicks and pops. The relatively smooth values of the "Hello" string would sound like a low, static-like rumble. The sudden jump from 0x00 (0) to 0xFF (255) would produce a very sharp "click" or "pop" because it's telling the speaker to move from its most retracted position to its most extended position in just 1/8000th of a second.
To get a birds eye view of the virtual ram system it maps all used virtual ram onto a space filling hilbert curve with selectable sub-section and tool-tips showing the proportional use of virtual RAM by every process. Right click any highlighted process on the map to 'Image' it's RAM in a new window. To be clear, this is virtual RAM mappings, and the numbers it gives are not useful when you're trying to figure out the physical amount of RAM any given process is using (though it is proportional).
On the image map there's a small "thumbnail" view for zipping around truly gigantic ram dump images for convenience on slower machines. The image of the RAM can animate it live at the user supplied rate (default every 1s) but this only really works with small amounts of RAM because refreshing big GB amounts takes time. When the RAM is 'Updating' the state it was when the 'Update' button was pressed is stored to compare against the final state when updating is done with the 'Diff' tool which turns green when there's an old version to diff against. The 'Strings' tool runs $ strings against the full RAM image or whatever set of offset selections are checkbox marked active and then marks them on the RAM image with a star and recenters the view on this location and displays all matches with their file offset in the text box. Deleting the search string and hitting enter on the empty box restores the raw strings output list.
[comment on this post] Append "/@say/your message here" to the URL in the location bar and hit enter.
I've been puttering away at improving my connmapperl port(s) of connmap (c/X11) geoip map network visualizer in terms of both features and performance. ip2asn.pl for the fully-offline(local) asn and org-name and peering look-ups is ever so slightly faster. The sql database side for long term detailed connection history is about 10x faster and 100x less IO which shows how bad it was before stateful connection tracking. Fancy things like sql queries and a mapping of the points onto the ipv4 range on a space filling hilbert curve (like the xkcd comic) also exist. The client program is forwards and backwards compatible; it just handles sudden socket closings a bit better, re: reconnecting now.
The size is from the included geoip dbs and regional internet registry whois dumps. Leave the .pl files in the extracted folder and just run them there.
All this started with and was forked and ported from an inspiringly simple .c program for X11 called connmap. My versions just happens to be in another language and graphical toolkit and tacks on sql points history and a lot of not necessarily useful features and eye candy. I still don't understand the original connmap's math for the lat/lon gps to x/y pixel translation but luckily through open source licensing I can build with it.
connmap-(tk|gtk2|gtk3) and [connmapperl-server + connmapperl-client] are a set of geoip map visualization applications. They rapidly call 'ss' and 'lsof' for established tcp connections and shows geoip dots, program process name, ASN, and Org Name where the remote peer is. They also can right click on points to pop up a window with copyable-text and a form for doing offline whois look-ups. It does this only accessing database files stored on disk. No network calls or getting banned by whois servers. It was made with lots of gemini-2.5-pro LLM AI help.
The size is from the included geoip dbs and regional internet registry whois dumps. Leave the .pl files in the extracted folder and just run them there.
The above are my simple ports and expanded forks from C to Perl of the X11 program connmap.
I did this because my Ubuntu 10.04 computer with Xorg 1.7 was too
old to run the original connmap program. At least 1 of these 5 programs
will run on almost anything from 1998 to 2025.
But as I did it I was having lots of fun so I added new obvious features and eventually diverged quite far from the base connmap code. Except in terms of lat,long to x,y calculation and the core concept; the hard stuff. That is still the same wonderful connmap logic.
"connmap-tk", "connmap-gtk2", "connmap-gtk3", and "connmapperl-server" have some extra features but don't
have the ability to scale like connmap. "connmap-simple-gtk3" is a pure port with no extra features that
can't scale like connmap either. None support "connmaprc" and instead use "connmapperlrc"; a new format.
"connmap-gtk2", "connmapperl-server", and "connmap-gtk3" are the best versions (though sometimes -gtk3 lags behind in features). They all load local/peer port map/color rules from config files which can be toggled on/off in the program by clicking the square buttons next to their name in the color key. They save all geoip points history, toggle with 'h' key, and can overlay them as white transparent circles scaled proportional to the number of times they appeared. They can save 's' and load 'l' this history file from disk. They can turn off global ip->asn/org-name look-ups and display globally by pressing 't'. They can right click near points and get a detailed list of their info as well as a field for doing full off-line whois lookups.
"connmap-simple-gtk3" is a direct port of connmap. No extra features. "connmap-tk" is somewhere in between.
"connmapperl-server" and "connmapperl-client" are the same program as "connmap-gtk2" but split into a gtk2 server/GUI and lightweight cli daemon clients which connect over the network via tcp/ip. It's really neat seeing all my computers and my VPSs active internet connections on one map at the same time on my home desktop. Since the client is pure perl and core modules only it runs on anything that has linux's "ss" and "lsof" programs.
Feature comparisons and Non-core Perl Dependencies
--- client/server architecture ---
"connmapperl-server" is a windowless GUI program server that listens on a port for client data and displays.
listens on port 6789
visual updates a tiny bit slower (1s), but client->server data updates are still fast (0.1s).
uses Gtk2 (which rocks) which your modern OS probably doesn't have.
can be dragged around with the mouse left-click hold.
has right-click map to get list of points IPv4 info within circle near the mouse pointer.
- optional ip2asn non-network based fully offline whois lookups for right click menu
has 't' key toggle for doing the 'text' ASN/Org-Name look-ups and showing them.
has 'h' key toggle for 'history' which shows all points seen scaled by count.
has 's' and 'l' keys to 'save' and 'load' history points.
has 'Esc' key to close and also an invisible exit button in the top right.
loads ports/colors/names from config file "connmapperlrc"
- does search/shows text depending on local/peer and port in "connmapperlrc"
- can turn look-ups for rules on/off by clicking square next to their name in the key.
- has 'c' config key to reload connmapperlrc file settings while running.
Perl Dependencies: use Gtk2, use Cairo, use Glib, use Net::IP, use Text::CSV
"connmapperl-client" is a tiny perl core modules only script that will run on everything, even routers.
is headless and uses a tcp connection to connect to connmapperl-server
requires the presence of $ ss and $ lsof binaries.
connects to port 6789
Perl Dependencies: none
--- single program architecture ---
"connmap-gtk2" is windowless and can be dragged around with the mouse left-click hold.
has right-click map to get list of points IPv4 info within circle near the mouse pointer.
- optional ip2asn non-network based fully offline whois lookups for right click menu
has 't' key toggle for doing the 'text' ASN/Org-Name look-ups and showing them.
has 'h' key toggle for 'history' which shows all points seen scaled by count.
has 's' and 'l' keys to 'save' and 'load' history points.
has 'Esc' key to close and also an invisible exit button in the top right.
uses Gtk2 (which rocks) which your modern OS probably doesn't have.
loads ports/colors/names from config file "connmapperlrc"
- does search/shows text depending on local/peer and port in "connmapperlrc"
- can turn look-ups for rules on/off by clicking square next to their name in the key.
Perl Dependencies: use Gtk2, use Cairo, use Glib, use Net::IP, use Text::CSV
"connmap-tk" has a traditional window bar.
hard coded: shows process name : port, ASN, and Org Name text for most ports
is Tk. Tk never changes and is available on both old and new OSes.
is Tk and so a little slower.
Perl Dependencies: use Tk, use Net::IP, use Text::CSV
"connmap-simple-gtk3" is a *direct* 1-to-1 connmap X11 feature port.
is a simple port that just shows red dots.
is what will run on most modern OSes.
is windowless and can be dragged around with the mouse.
Perl Dependencies: use Gtk3, use Cairo, use Glib
"connmap-gtk3" is windowless and can be dragged around with the mouse left-click hold.
has right-click map to get list of points IPv4 info within circle near the mouse pointer.
- optional ip2asn non-network based fully offline whois lookups for right click menu
has 't' key toggle for doing the 'text' ASN/Org-Name look-ups and showing them.
has 'h' key toggle for 'history' which shows all points seen scaled by count.
has 's' and 'l' keys to 'save' and 'load' history points.
has 'Esc' key to close and also an invisible exit button in the top right.
uses Gtk3 which is what will run on most modern OSes.
loads ports/colors/names from config file "connmapperlrc"
- does search/shows text depending on local/peer and port in "connmapperlrc"
- can turn look-ups for rules on/off by clicking square next to their name in the key.
Perl Dependencies: use Gtk3, use Cairo, use Glib, use Net::IP, use Text::CSV
The local/peer ports, their names, and their colors from a config file located in the same directory as it: connmapperlrc. These rules can be toggled for IP->ASN/Org-Name resolution and display by setting the last field to 0=Off or 1=On, or done so while the program is running by clicking their corresponding colored square in the key.
# connmapperlrc - Configuration for ConnMapperl
# Format: KeyLabel,Type,Port,Red,Green,Blue,Alpha,ShowText
# ShowText: 1 = Display dot and text, 0 = Display dot only
"Webserver",local,80,0.2,0.5,1,0.8,1
"Webserver",local,443,0.2,0.5,1,0.8,1
"Web Surfing",peer,80,0.68,0.85,1.0,0.8,0
"Web Surfing",peer,443,0.68,0.85,1.0,0.8,0
"IRC",peer,6667,1,1,0.2,0.8,0
"IRC",peer,6669,1,1,0.2,0.8,0
"IRC",peer,6660,1,1,0.2,0.8,0
"IRC",peer,5555,1,1,0.2,0.8,0
"SSH Out",peer,22,0.6,0.2,0.8,0.8,0
"SSH In",local,22,0.7,0.2,0.6,0.8,0
"IMAP/S",peer,993,0.1,0.9,0.2,0.8,0
"IMAP/S",local,993,0.1,0.9,0.2,0.8,0
"smtpd",local,587,0.3,0.6,0.3,0.8,0
"Torrents",local,12000,1.0,0.4,0.7,0.8,0
"Torrents",peer,12000,1.0,0.4,0.7,0.8,0
"Shadowsocks",peer,31415,0.5,0.5,0.5,0.8,0
"Shadowsocks",local,31415,0.5,0.5,0.5,0.8,0
"Shadowsocks",peer,31416,0.5,0.5,0.5,0.8,0
"Shadowsocks",local,31416,0.5,0.5,0.5,0.8,0
"connmap",local,6789,0.5,0.5,0.1,0.8,0
"connmap",peer,6789,0.5,0.5,0.1,0.8,0
# By default only connections to the local webserver are shown with ASN+OrgName lookups
# because that process is somewhat cpu and io intensive. This applies to the 'h' history
# function too. Points matching local or peer ports with ShowText field 0 will not show
# up as dots in history's scaled dot display.
# Generally anything that's always going that you already know about set ShowText 0 to
# ignore and not blow out the history heatmap.
Right-click the map to get a selectable text pop-up of IPs near the click. The white circle which appears briefly shows the range. Local whois/peering look-ups available in the pop-up window detail list.
There a lots of comments at the top of the source re: configuration required if any.
Paths are set relative to the folder so as long as ./resources/ exists
and you have the perl modules required it should work.
An aside on the lack of libgtk2-perl on many distros
There are a few tricks to getting it to work but the most important one is at the very top of the ./libgtk2-perl/debian/rules file (on a new line before any other text), adding the following line:
Without this it's all "xs/Gtk2.c: loadable library and perl binaries are mismatched (got handshake key 0xce00080, needed 0xed00080)" errors when Gtk2 is actually attempted to be used.
Otherwise it was just a matter of changing some paths and versions in the ./DEBIAN/control file of the libgtk2-perl_1.24993-1_amd64.deb produced after, "$ DEB_BUILD_OPTIONS=nocheck debuild -us -uc -b" like changing DEBIAN/control so that perlapi-5.28.1 -> perlapi-5.32.1, and changing the folder name of ./usr/lib/x86_64-linux-gnu/perl5/5.28 to ./usr/lib/x86_64-linux-gnu/perl5/5.32 (for Debian 11, for other distros use the appropriate perl version/path).
I did this by unpacking the produced .deb,
dpkg-deb -R libgtk2-perl_1.24993-1_amd64.deb tmp/
Then doing the above changes manually, then repacking,
dpkg-deb -b tmp libgtk2-perl_1.24993-1_GOODPATHSWORKING_amd64.deb
Then there's the command line tools I made for offline whois.
If connmap-gtk2.pl detects ip2asn.pl in the directory it will offer the option to use it for all local (no network connectivity needed) look-ups of whois dumps from the regional internet registries.
"ip2asn" is a sort of janky fully-local "whois" for all 5 regional internet registries.
The other scripts are for processing the raw data to a form usable for ip2asn.pl
It started as a scratch pad for testing which were the best ways to go from IP to
organization name but eventually became a useful tool in it's own right to avoid
getting throttled by internet whois servers when doing many lookups in a row.
It uses 4 diferent methods to try to go from IP to ASN and org name.
The first 3 methods are relatively fast.
The 4th, awk'ing through big whois dumps, is not.
It is called like: $ ./ip2asn 94.74.97.164
--- Found IP Information for: 94.74.97.164 ---
Registry: apnic
Country Code: SG
Start IP: 94.74.64.0
Range Size: 16384
Status: allocated
Opaque ID: A917E678
--- Associated ASN Records ---
HUAWEI INTERNATIONAL PTE. LTD.
ASN: 131444, Count: 1, Status: allocated, Date: 20160715
ASN: 136907, Count: 1, Status: allocated, Date: 20170807
ASN: 141180, Count: 1, Status: allocated, Date: 20200914
ASN: 149167, Count: 1, Status: allocated, Date: 20211206
ASN: 151610, Count: 1, Status: allocated, Date: 20230712
HUAWEI INTERNATIONAL PTE. LTD.
22.2908 , 114.1501136907 - "HUAWEI CLOUDS"
aut-num: AS131444
as-name: HIPL-AS-AP
descr: Huawei IT Data Center in AP
country: HK
remarks: --------------------------------------------------------
remarks: To report network abuse, please contact mnt-irt
remarks: For troubleshooting, please contact tech-c and admin-c
remarks: Report invalid contact via www.apnic.net/invalidcontact
remarks: --------------------------------------------------------
org: ORG-HIPL2-AP
admin-c: HIPL7-AP
tech-c: HIPL7-AP
abuse-c: AH905-AP
mnt-lower: MAINT-HIPL-SG
mnt-routes: MAINT-HIPL-SG
mnt-by: APNIC-HM
mnt-irt: IRT-HIPL-SG
last-modified: 2020-06-17T13:05:46Z
source: APNIC
It doesn't always work that nicely though. And lacnic entries are pretty bare.
All this is done by including 600 MB of RIR ipv4 delegation files,
all five regional internet registry's whois and peering dumps,
various maxmind geolite ip->lat,long csv databases, and
caida.org/archive/as2org/ as-org2info.txt. See appendix for URLs
Appendix of various resources and notes
### the various resources/* files and where they come from to update them.
## RIR whois/peering db
# RIPE NCC https://ftp.ripe.net/ripe/dbase/split/ripe.db.aut-num.gz
# ARIN https://ftp.arin.net/pub/rr/arin.db.gz
# APNIC https://ftp.apnic.net/apnic/whois/apnic.db.aut-num.gz
# LACNIC https://ftp.lacnic.net/lacnic/dbase/lacnic.db.gz
# AFRINIC https://ftp.afrinic.net/dbase/afrinic.db.gz
## RIR Delegation files
# https://www-public.telecom-sudparis.eu/~maigron/rir-stats/
# https://ftp.afrinic.net/pub/stats/afrinic/delegated-afrinic-extended-latest
# https://ftp.apnic.net/stats/apnic/delegated-apnic-extended-latest
# https://ftp.arin.net/pub/stats/arin/delegated-arin-extended-latest
# https://ftp.lacnic.net/pub/stats/lacnic/delegated-lacnic-extended-latest
# https://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-extended-latest
early version of the gtk2 port, what the gtk3 port looks like
recent version during a distributed botnet attempting to mirror the entire website at once
connmap-tk with integrating time lapse mode enabled by commenting out the "connection_item" tag on the dots:
# 1. Draw the dot using Tk::Canvas->createOval
$canvas->createOval(
$x - $radius, $y - $radius, $x + $radius, $y + $radius,
-fill => $dot_color,
-outline => $dot_color, # Use same color for outline to make a solid dot
# -tags => 'connection_item',
);
It's easy to see there is some error in the geoip placement on the map gif background.
[comment on this post] Append "/@say/your message here" to the URL in the location bar and hit enter.
It is creating a problem, and fragmentation, for program configuration where none existed in exchange for the chance of a slight speed-up by keeping everything in ram. Setting this as a default in Debian 13 signals the Debians defaults are for *big* machines with excess ram. Users with smaller amounts will have to do a post install manual configuration to achieve normal performance and avoid slowdowns from tempfs taking ram and pushing things into swap when the ram inevitably fills. And the fallout from programs having to change their code just to avoid using /tmp for what it's meant for will be much larger.
Now you may say, "once there's memory pressure the tempfs will push back to disk anyway via swap" and that is true. But it is also true that the memory pressure wouldn't exist without the tempfs. And this pushing back and forth can only slow things down when compared to a more straightaway, less steps/io, setup of just files in folders on disk.
Another objection is "most people don't have much stored in /tmp". And that's also true. But that doesn't change the fact that /tmp is there to be used. It's meant for 5kb or 5GB. There has never been a limit. And now people are trying to say there must be a limit because of the bad downsides of this change.
That's not to say there are no upsides. The performance increase in some applications, like Firefox, is notable. Considering this, /tmp should be left alone and a new /tmpfs in ram "standard" be created for programs that desire the potential speed up. That way existing workflows would not be broken and devs would not have to change their programs code unless they wanted to.
In this context the "ram is there to be used" is a bad application of the meme, a bad default, and the /tmp->tempfs change resulting from it should be reverted in future Debian 13 point releases.
The good thing is that it's an easy fix for users (if not software devs): "You can return to /tmp being a regular directory by running systemctl mask tmp.mount as root and rebooting."
[comment on this post] Append "/@say/your message here" to the URL in the location bar and hit enter.
superkuh on blog at (photoblog post, non-rss feed)
This 30ft*12ft solar hot air balloon envelope made up of 5 gores was literally too big to handle with just one person. After this photo with 2 people I took it out by myself on a 8mph wind day and it ripped after fully inflating. I later managed to tape 4 of it's remaining gores back together.
[comment on this post] Append "/@say/your message here" to the URL in the location bar and hit enter.
I took this photo while testing my solar hot air balloon. It's much bigger than it looks from this angle. It's filled with air by hand then the sun heats it up and it rises. It's made by cutting off a 30ft * 12ft piece of 0.3 mil painter's plastic drop cloth ($30/300ft), folding it in half and sealing the remaining edges with clear packaging tape. The pillow shaped balloon then has one corner cut off and a cardboard mouth/ring put in for filling and the tether. The dark coloration is from air-float charcoal ($15 1lb) put inside and shook around till it coated the surfaces. These types of "free balloons" can lift a good fraction of a pound to over 60,000ft. But until I can understand and fulfill the FAR 101 regulatory requirements fully these are just brief handheld tests below local structure heights.
I learned how to make this type of balloon from the excellent documentation at Danny Bowman's Bovine Aerospace website and youtube videos.
The cardboard ring holding the balloon mouth open for inflation can be folded down onto itself a couple times to close the mouth and prevent wind from deflating the balloon. This folded cardboard block also provides a good place to wrap and knot the tether to the balloon. It's also fairly air-tight so a long flat tube casually rolled up as a pressure release valve might be necessary for non-tethered flight.
First lay out 30ft of the unfolded sheet. Then unfold it sideways and fold it vertically to make it two joined overlapping 15ft long sheets. Weight it down with random soft objects. Then go around the remaining 3 edges taping them together.
Now that I've figured out construction and everything works I'll move on to making a proper large envelope with normal diamond-like gores joined with strapping tape to distribute the load.
[comment on this post] Append "/@say/your message here" to the URL in the location bar and hit enter.
superkuh on blog at (photoblog post, non-rss feed)
4-whistle.stl - whistles are the hello world of 3d modeling for 3d printing so I thought I'd try my hand to test my new printer. This '4 whistle' is 4 whistles all sharing the same mouth piece. 2 of of the same length and 2 are shorter or longer. So there are 2 whistles at the same tone beating against each other in sync and then two whistles at slightly different frequencies out of phase going in and out of beating interference.
It is not loud. Just blow softly. It sounds sort of like an old english police whistle.
20% infill and no supports at 0.2mm layers. ~1 hour. (Printables Page)
[comment on this post] Append "/@say/your message here" to the URL in the location bar and hit enter.
superkuh on blog at (photoblog post, non-rss feed)
3.5 cups bread flour, 1.25 cups water, 4 tsp salt, 1 tsp yeast, two periods of kneading a handful of hours apart. Boiled in salt water, washed, then added to broth. The broth is 4 tsp won-ton powder, 4 cups of water, a dash of sherry wine, 2 drops sesame oil, 1 tsp (lots!) of white pepper, 5 cilantro strands wrapped into a knot and soaked/removed, 1 jalapeano pepper split soaked and removed, 6 finely diced garlic gloves, and green onions garnish. The meat is pressure+slow cooked pulled pork in soy sauce/ginger/garlic/hoison sauce.
[comment on this post] Append "/@say/your message here" to the URL in the location bar and hit enter.
superkuh on blog at (photoblog post, non-rss feed)
A 5 gallon bucket with holes in the bottom and sides and an towel attached with zip ties and folded underneath makes for a decent improvised zamboni for smoothing out the surface of rough ice out on the rink I maintain(ed) on the lake this year.
[comment on this post] Append "/@say/your message here" to the URL in the location bar and hit enter.
