))
A change of perspective
Immutable Insight GmbH and I went separate ways after the end of March. We shared a compelling and demanding journey through the highs and lows of crypto markets in the last years.
I want to thank the founders and all of my colleagues for the delightful time given together. The opportunity to learn from all of you and give back a little bit of what I could offer made Immutable Insight a fantastic starting point for my non-academic career.
I started as consultant at d-fine GmbH with the beginning of April, and with the new job perspective I have high hopes on a little bit more writing. I can already tell that there's a lot of interesting, but unfinished articles which I really want to address and finish writing in the near future.
Python: Messing around with pyproject.toml and yapf
I'm finally springing on top of the pyproject.toml (PEP-517) train. Choo choo!
All whistles and bells on board: poetry, pre-commit and all revelant helper tools, e.g., yapf.
But - there's always a slightly annoying "but" - adding a pyproject.toml to my current project had the side-effect of stopping my pre-commit workflow:
yapf.....................................................................Failed - hook id: yapf - exit code: 1 yapf: toml package is needed for using pyproject.toml as a configuration file
Well, well. After a prolonged time of searching the web and finally understanding that neither pyproject.toml nor poetry were at fault, I found the culprit itself with pre-commit.
And, the solution was given to me by asottile:
- repo: https://github.com/google/yapf rev: v0.32.0 hooks: - id: yapf additional_dependencies: [toml] # Adding toml package as dependency for pyprojec.toml
The simplicity of the solution is again an embarassing way of saying to me: "I didn't care to comprehend the error message."
Next time I proof to be better!
Data protection regulation
Already in January 2022 the Landgericht München passed the judgment that externally loading Google Fonts exposes your dynamic IP address.
For the time being I'm relying on loading the MathJax Javascript library from a CDN, but only on those sites which require MathJax, e.g., when using mathematical formulas. Fortenuately, in my case, the same CDN which provides MathJax also caches my webpage. To make things a little bit more clear, I added a link to Cloudflare's privacy policy.
For the foreseeable future, I plan to deploy the whole webpage on a private server including a MathJax copy such that I fully control the sources. However, I will likely rely on Cloudflare DDoS protection and cache to enhance the reliability of the webpage (and survival of my server).
Apple MacOS: Conda in Tmux
I need to run some lengthy processes, but MacOS is getting in my way by hibernating my Macbook Pro after some time. And, of course, I tried to get my hands on the energy saving settings. However, it seems that I am lost at it as not a single of my settings seem to help. Or, I just don't understand them - maybe next try, but not today.
As a quick solution, I just use nohup for single commands, therefore, the command keeps running after a hibernate.
But, ultimately, I want to use tmux; it's already installed, but I have found that tmux and conda mess on MacOS with each other.
I can not run my Python environments in tmux!
It always defaults to the systems Python executable.
A quick search and I find two solutions:
Well, both provide a solution to my issue, and to some extent explain it. But, why? Why can't we have nice things…
Apple MacOS: Spotlight
I'm new to the wonderous world of Apple Mac products. And, with an instant, I describe my experience as:
If you really do not want to care about your work/computing environment, then, an Apple will be fine.
However, I deeply care for my work environment, I'm always trying to improve my tooling. Hence, why I prefer to use a Linux-based environment, e.g. Archlinux or Linux From Scratch. On the other side, Apple's idiom seems to be that they will care for ones tooling, not oneself. And, I have some issues with such an idiom and its consequences:
- I do not know what is running.
- I do not know when it is runnning.
- I do not know how to fix it - the documention is hard to reach.
And that is my problem with Spotlight, it's a fancy index search/program starter.
At hand, it's indexing my heavy working directory, e.g., where I place millions of files every day - in that, AppleFS would be worth another story.
How did I find out that I have a problem with Spotlight? Thousands of running threads, heavy CPU load, etc, which throttle the performance of my MacBook.
But, the only thing I know is a service name mdworker_shared. Favorite search engine to my help!
So, Spotlight runs an indexing services mdworker_shared in the background, which would be fine for a standard user with a low increasing number of files.
Spotlight allows to exclude directories/files under the name of data privacy, and with that, my device is running smoothly (again).
PS: I also needed to exclude my working directory from Time Machine…
Postfix: Smarthost and Strato
You may or may not have noticed that I moved my webpage away from DESY as I left DESY for my job at Immutable Insight with the beginning of September. With that change at hand, I used the chance to get myself an own domain: https://phibra.de - with all the bells and whistles.
So far, I'm quite happy with my new setup - I may elaborate on the details of it in a future article. So now, I combine Amazon S3 storage as a static webhost with Cloudflare's caching to slash costs, and Strato as domain reseller and mail server provider.
If you would ask me now what is the biggest earn of the new setup?
I would answer that I can specify an own MX record for my domain.
No dependence on whatever mail service you could call out.
However, setting up and running a mail server is a pain in the ass.
Therefore, I lied when I said that I do not depend on any mail service server.
I use Strato's mail server.
Especially, I relay every mail from my private servers using Postfix to the Strato mail server.
Lo and behold, like everytime I work on a mailing system, there are some issues with it and I have seldomly a clue why. I mean I know what I am doing and I really try to understand the errors, but understanding and fixing mail errors is sometimes like hell on earth.
With some little help from my favorite web search engine, I came up with the solution - in the end: First, we need to inspect our connection with Strato's mail server, where I used the great instructions from Steven Rombrauts. As we use TLS, we spin up openssl and connect to the mail server using the default SMTP port:
sudo openssl s_client -starttls smtp -connect smtp.strato.de:465
No chance to connect to the Strato server, connection closed.
First thing then, I checked with the options -state -debug the details of the connection.
And, I needed to fix my certificate chain:
sudo openssl s_client -starttls smtp -connect smtp.strato.de:587 \ -state -debug \ -cert /etc/letsencrypt/live/<DOMAIN>/cert.pem \ -key /etc/letsencrypt/live/<DOMAIN>/privkey.pem
After fixing (?) my certificate chain, I still could not connect to the mail server.
And then, it hit me! Strato configuration !!!
I had to read that page twice - mea culpa, I didn't expect that information to be lying around in plain view:
Bitte beachten Sie: Bei der Verwendung als Smarthost (Relay), bspw. über Exchange 2003, verwenden Sie bitte die Standardauthentifizierung per TLS (STARTTLS) über den Port 587 (alternativ Port 25). Zur Authentifizierung verwenden Sie bitte wie gewohnt Ihre E-Mail-Adresse sowie das dazugehörige E-Mailpasswort.
Die Nutzung als offenes Mail-Relay ist nicht möglich, da ausschließlich das SMTP-Auth-Verfahren unterstützt wird.
Yep, yep, I used the port 465. Changing port 465 to 587 did the trick.
Embarrasing.
Matplotlib Cheatsheet
Today, I came across the magnificent cheatsheet from Matplotlib: https://github.com/matplotlib/cheatsheets#cheatsheets.
And, it is awesome! In general, Matplotlib is great, the API, the documentation, the tutorials, the examples, and so on. The developers and the community put a lot of effort into the project, and that effort is showing off.
The cheatsheet is rather new, the earliest commit in the Github repository is from , hence, the existence of the cheatsheet got me by surprise, today.
However, I can tell you the design and content of the cheatsheet goes very well. It is a great addition to just look at and draw some inspiration.
Update: Although, there has been submissions to Hackernews (https://hn.algolia.com/?q=matplotlib+cheatsheet), none of the submissions took off (having enough points to reach the "front" page). I should crawl the new page more often.