Michael Driscoll's Blog, page 8

PyDev of the Week: Laysa Uchoa

This week, we welcome Laysa Uchoa (@laysauchoa) as our PyDev of the Week!  Laysa is very active in the PyLadies group and gives many talks at various conferences.

You can catch up with Laysa’s work by checking out GitHub profile.

Let’s spend a few minutes getting to know Laysa better!

Can you tell us a little about yourself (hobbies, education, etc):

I come from a rural area of Brazil. Where I grew up, we would face weeks without electricity, and some neighboring cities had none. I guess this is the first technology that changed my life. This motivated me to pursue my Bachelor’s in Electrical Engineering.

This was great because it opened my mind to programming – a powerful way to control devices. I started with low-level programming and eventually moved on to high-level languages.

Now, I’m at the highest level with Python.

Besides programming languages, I love human languages. I speak Portuguese as a native, along with English, Spanish, and German.

Why did you start using Python?

I started using it at the university for research. We needed an open-source language that could analyze huge amounts of data. Python was the best candidate back then and still is.

What other programming languages do you know and which is your favorite?

In the university, I learned Verilog, C, C++, and Python. Later, I worked a bit with Rust, sometimes Javascript, but mainly with Python. I use a fair amount of bash in my day-to-day work as well.

Python has been a constant in my life, and also my favorite language.

What projects are you working on now?My presentation about Serverless with OpenSearch for PyCon Sweeden 2023PyLadies Munich 9th anniversary meetupWhich Python libraries are your favourite (core or 3rd party)?datetime because time mattersos and sys for automation loversrequests to be chatty with APIspandas for data science topics How did you get involved with the PyLadies group?

When I moved to Munich, I used to visit a lot of local Python groups. They are great, but there were not so many women visiting the groups. I thought it would be nice to have a group where we could go and talk about how is our work life and the amazing things we can do in the Python environment. So, together with others, we reactivated PyLadies Munich.

What was the hardest talk you gave and why?

For me, it was when I gave a talk about careers in software development to an educational TV channel in Brazil. It was live-transmitted to thousands of students from the public educational system.

These students were working very hard to secure a place in a public university. For many of them, this is a life-changing opportunity, just as it was for me. This talk wasn’t all about careers, it was about life. It doesn’t matter where you come from; dreaming big is something everyone can and should do. My message to them is that knowledge is this magical thing that can transform our dreams into reality.

This talk holds a special place in my heart because, in the past, I was one of those students too.

Is there anything else you’d like to say?

Absolutely!

I want to encourage everyone, especially those who are just beginning their journey in tech or Python, to maintain their curiosity and keep exploring. Technology is a vast and thrilling field with endless opportunities for learning and creation.

And here’s my message to all the dreamers and technologists out there: If you can dream it, you can make it happen with Python!

Feel free to connect with me on Twitter, where we can chat about all things Python: https://twitter.com/laysauchoa

Thanks so much for doing the interview, Laysa!

The post PyDev of the Week: Laysa Uchoa appeared first on Mouse Vs Python.

The Ruff Formatter – Python’s Fastest Formatter!

The Ruff project was released about a year ago and stormed the Python world. Ruff is written in Rust, but lints Python super fast. You can read all about it in An Intro to Ruff.

This week, the Ruff formatted was released! You can still use Ruff to lint your Python code, but now you can also use Ruff to format your code with 99.9% Black compatibility. The good news? It’s over 30x faster than Black!

Getting Started

If you haven’t got Ruff already, you’ll need to install it.

pip install ruff

If you do already have Ruff, be sure to upgrade to latest so you can give the formatted version a try!

Formatting with Ruff

To format your code with Ruff, you need to run ruff format.

Here’s an example of formatting your current folder:

ruff format .

The official announcement for the Ruff formatter shows it formatting Django, Zulip and other Python projects in less than a second.

Ruff is Configurable

The Ruff formatter adds some configuration options that are absent from Black. Namely, you can tweak your preferred quote and indentation style:

[tool.ruff.format]
quote-style = "single"
indent-style = "tab"Wrapping Up

Ruff can now replace flake8, pyLint, Black, isort and more. Try out the Ruff formatter today! You’ll like it and it will speed up your development time too!

The post The Ruff Formatter – Python’s Fastest Formatter! appeared first on Mouse Vs Python.

PyDev of the Week: Simon Cross

This week we welcome Simon Cross (@hodgestar) as our PyDev of the Week! Simon lives in South Africa and helped start PyConZA. You can see all the various open-source tools that Simon works on over on GitHub.

Let’s spend some time getting to know Simon better!

Can you tell us a little about yourself (hobbies, education, etc)

I’m a South African who grew up around Melkbosstrand and Fish Hoek which lie on the northern and southern ends of the beautiful Cape Town.

I enjoyed coding from an early age and after studying physics, mathematics and computer science at university moved on to a varied succession of software jobs. Highlights include working on bioinformatics at SANBI which was my first full-time Python job, helping build KAT-7 an early prototype for the upcoming SKA radio telescope, a stint at REACH and running a small data science team at Prodigy Finance.

Recently I combined by physics roots and my love of Python and became involved in writing Python tools for simulating and controlling quantum devices, which lead to my current role at Zurich Instruments developing quantum control software.

Why did you start using Python?

I didn’t have a particularly deep reason. I was working in a few languages for different projects. Python was just a really nice language to work in.

It was well thought out, self-consistent and very readable. It had less boilerplate than Java and it didn’t allocate half your RAM to itself on start up. Unlike C, there are nice tools like lists and hash tables (i.e. dicts) available out of the box. It was great for small projects, but also scaled really nicely as a code base grew. It wasn’t just for the web.

These are still good reasons to use Python today.

Another factor was that I had a bunch of friends who also liked it. We had a lot of fun coding things together, starting meet ups, talking about computer science, technology, the world, life, … .

In particular my close friend Neil Muller was my partner in crime in many Python (and non-Python) projects including CTPUG, Sutekh, Wafer and numerous PyWeek entries.

It’s strange to think that Python wasn’t particularly accepted for a long time. Professional software engineers from big institutions would say that Python wasn’t a proper language, people worried that there might not be jobs for Python developers. Variations on these old memes still float around today, despite Python’s incredibly wide and successful adoption.

What other programming languages do you know and which is your favorite?

I use C and Javascript regularly and have done some bigger projects in both.

Javascript is of course ubiquitous for coding web applications, and I use it there, but it’s also not a terrible language for writing services in.

C is a really great language. It sits at sweet spot in the abstraction layers where one is separated from many of the details of how the CPU works and yet still describing very precisely the operations to perform.

What projects are you working on now?

I’m involved in HPy and QuTiP at the moment.

HPy is a new API for writing Python C extensions like numpy. Python’s old C API exposes most of Python’s internals which makes it hard to provide a stable target for C extensions to build and link against while still allowing Python itself to evolve.

HPy is nearing its 1.0 release and there was a lot of discussion of HPy at the Python language summit this year, so I think it’s likely that we’ll start to see HPy used and adopted more widely.

QuTiP is a Python toolkit for simulating open quantum systems. Here “open” means a system that isn’t isolated and interacts with it’s surroundings — so really all quantum devices that one controls and interacts with. QuTiP is used by many academics and companies working with quantum devices.

I was lucky enough to be working on QuTiP full-time for a couple of years, but now I’ll be doing work on it as my spare time allows.

I’ve just started working at Zurich Instruments on LabOne Q. LabOne Q is open source, and we’re hoping to grow the community around it in the future, but for now I’m still learning the ropes.

Which Python libraries are your favorite (core or 3rd party)?

My favourite core Python module is “import this”. It doesn’t do anything, but there is a lot of software engineering wisdom encoded there, and I find myself going back to it every couple of years and reading it again. I’ve also had all of the lines printed on mugs for PyConZA (PyCon South Africa) or CTPUG (Cape Town Python User Groups) a few times.

Outside of the core libraries, I’m a big fan of the many libraries which had a significant impact on Python’s direction, even if they didn’t end up becoming massively adopted themselves or weren’t even Python libraries. For example, the Twisted asynchronous networking library eventually led to the creation of asyncio and language level support for asynchronous patterns. PyPy showed how great Python performance could potentially be, and I think has inspired many of the other attempts that have come and gone. Numpy and Pandas both created huge new communities of Python users, and I’m hoping that projects like PyScript and BeeWare will do the same for Python in the browser and on mobile in the future.

How did you end up starting PyConZA?

I attend PyCon US for the first time in 2012. I was working for REACH at the time — it was called Praekelt Foundation then — and on the way back my boss, Simon de Haan, messaged me to say, “We should start PyCon South Africa, do you want to do it?”, and I said yes.

I’d organized quite a few smaller events before, but I really knew nothing about organising a full conference. We had to find speakers, sponsors, a venue, sell tickets, advertise, have a budget and these were all new to me.

I’m very grateful to Praekelt for supporting PyConZA during the first few years, both by letting me spend a fair bit of work time on it, but also financially,

I’m also very grateful to Lauren Kotze and Sam Manclark who both gave a lot of their time and wisdom to help with the organization even though they weren’t Python developers at all. Lauren especially stayed heavily involved behind the scenes all the way until PyConZA became an independent non-profit many years later.

Organizing the first year was crazy, but a lot of fun because everything was new. Some of the later years were much harder. It wasn’t new, so there was a bit less excitement, funding was tighter. Eventually it started to get a bit easier again, but that took a few years.

Do you have any tips for others who want to start a meetup or conference?

I don’t claim much special wisdom around organizing events or communities, but I can share some things that worked for me or I wish I had been more intentional about:

At the start you’ll have no experience to guide you, so just start. You’ll make mistakes and learn on the way.If you’ve never organized anything, start small so that the mistakes are smaller too.Event organizing is a separate field all on its own, so if you’re primarily a software developer, you likely don’t know much about it. Find some professionals to talk to even if you can’t afford to hire them.If you take on something big like organising a full conference, it’s going to be hard sometimes, so it really helps to have someone else as committed as you are to share the load. Even just one other person is a huge help.Try be clear with yourself what your big picture goal is and what you and the other organizers are getting out of it. These will likely both change from year to year, but don’t look up a three years later and wonder if it was worth the sacrifices.If an community is bigger, there will be many people within it who have opinions you disagree with, and many people within the community who disagree with each other. Think a bit about how you as an organizer want to approach this.

Thanks so much for doing the interview, Simon!

The post PyDev of the Week: Simon Cross appeared first on Mouse Vs Python.

PyDev of the Week: Pradeep Kumar Srinivasan

This week, we welcome Pradeep Kumar Srinivasan as our PyDev of the Week! Pradeep works on the Pyre package, which is a performant type-checker for Python as well as the Pysa security tool.

You can sometimes find Pradeep at PyCon giving talks and interacting with the community.

Let’s spend some time getting to know Pradeep better!

Can you tell us a little about yourself (hobbies, education, etc):

From childhood, I have been obsessed with one question: what is the best way to think? Accordingly, I devour books on rationality, cognitive biases, and automated reasoning. My college blog was called “Road to Rationality”, and I’m still trying to travel, hopefully faster, on that long road.

My hobbies include playing badminton, going on long bike rides, and learning about the tectonic plates of economics and culture that shift surface-level phenomena in the world. I’m also a huge fan of P. G. Wodehouse and have spent many a pleasant day poring through his prose and marveling at his similes.

I did my Bachelor’s in India and Master’s in the U.S., both in Computer Science, and moved to the Bay Area for work.

Why did you start using Python?

I first came across Python as an undergrad. Until then, programming for me had been synonymous with C/C++, which I had found pretty painful. When I was doing some course projects, though, I found that I had the option of using Python. I couldn’t believe the professors were letting us get away with using a language as fun as Python for serious assignments. No manual memory management, no pesky pointers, and, what seemed to be, in comparison to C/C++, no rules at all! It seemed almost too good to be true. Even afterwards, I somehow still thought that, while I could use Python for my own hobby projects, professional projects had to be in serious languages such as C++ or Java.

That finally changed when I spent a summer on the Django web server for my university’s massive, student-run technical festival (called Shaastra), where we gave the organizers tools to track attendees and resources. That’s when I dropped the “serious vs fun languages” dichotomy. Just as with Python, I was struck by how clean and pleasant Django was. It was a great education in how a program designer could apply the “Don’t Repeat Yourself” principle beyond just de-duplicating identical lines of code. More generally, it showed me that you don’t need to sacrifice elegance for practicality.

What other programming languages do you know and which is your favorite?

I followed Eric S. Raymond’s advice and learned Lisp as my second language. It really was an enlightening experience to solve the exercise problems from Structure and Interpretation of Computer Programs and start to get The Way of functional programming. I still hack on Emacs Lisp at home for personal productivity.

Over time, though, I’ve been seduced by the wonders of static typechecking. (Very roughly, “static typechecking” means getting so-called “compilation errors” up front, just as we do in C++ or Java, even in a dynamic language such as Python.) These days, I can’t even imagine confidently changing a non-trivial program without typechecking.

I love Haskell for its purity and expressiveness, though I find the compilation time and dependency hell frustrating. When working on the Pyre typechecker, I’ve been using OCaml, so that’s the functional language with which I have the most professional experience, and I have enjoyed how it lets you express complicated code without sacrificing speed or type-safety, especially using modules and module functors. I have also spent a few months writing Rust code on the Mozilla Servo project.

When it comes to everyday coding, though, I think it’s hard to beat Python. It has pretty much every library I want and it just works. Python’s biggest superpower may, weirdly enough, be its syntax. In almost every other language, I miss the significant indentation (and, thus, the lack of ugly braces) as well as the sweet syntax for comprehensions/decorators/with-statements. Keyword-arguments, for example, are prosaic and easy to underestimate, but I miss them sorely when I need to modify a years-old PHP or JavaScript function call and can’t puzzle out which of the arguments correspond to which parameters. (I’ve seen people try to work around this by using comments to indicate the keyword-argument name in the function call; that’s just a bug waiting to happen when the function changes.) The more the years go by, the more I appreciate how, despite not being the most expressive language around, Python stays competitive with its syntax, ecosystem, and pleasantness.

Which Python libraries are your favorite (core or 3rd party)?

I’ve found libcst to be indispensable for quickly analyzing or modifying Python code. For example, I’ve used it to collect detailed stats on how people are using type annotations in the wild. I couldn’t have done that so easily without libcst.

Other favorite libraries are pytest for testing, itertools/more_itertools for common transformations, and Beautiful Soup for extracting information from web pages in my personal scripts.

How did you get involved with the Pyre project?

I joined Meta (which was then Facebook) after grad school, following a summer internship there. I had done a lot of projects on static analysis and I was looking specifically for language-level teams. I spoke to a few teams and found Pyre, which was a great fit for me, as I got to do work I like (static analysis) on a fun language (Python) using another fun language (OCaml).

One great benefit of having a large internal user base was that I got to field hundreds of questions and bug reports. This made me dive into Python’s docs and PEPs to understand both its runtime behavior and, of course, its static type system. There’s nothing quite like an urgent bug report to motivate you to discover the obscure clause of some old PEP that specifies the correct behavior. I also got to see exactly how existing features were being used in our code base, where the most pressing gaps were, and thus which new Pyre features would most help our users.

Another benefit was getting to work closely with the open-source Python typing community and actually contributing back to the typing standard. Thanks to the encouragement of the Pyre team members, the typing community, and my collaborators, I was able to contribute PEP 646, PEP 673, and PEP 675. I was also able to give talks about my work at PyCon 2022 and PyCon 2023, as well as a PyCon Tutorial on “Python Types for Fun and Profit” (with Pyre teammates Shannon Zhu and Jia Chen).

As Pyre has matured, I’ve started branching out beyond catching “compilation errors” (aka type errors), and have been trying to use the typechecker as a platform for other analyses, including privacy enforcement and code-navigation (such as go-to-definition).

What projects are you working on now?

As part of using the typechecker as a platform, I’ve been working on a feature in Pyre called ReadOnly. You can roughly think of it as declaring a Python variable to be “deep const“. This means that not only can you not mutate any of an object’s direct attributes, you cannot mutate any of the attributes’ attributes or call any methods that may mutate the object. Using this feature, you can guarantee that code will not write data to places where it shouldn’t. For example, if you have a function that accesses sensitive data and you don’t want to leak that sensitive data to any of the function’s parameters by mistake, you can mark the parameters as ReadOnly so that they can never be written to.

Besides programming, I also run a monthly Typing Meetup. When working on some of my PEPs, I saw that there were many ideas that excited people but that no one was actively working on, or that were getting bogged down in long mailing-list threads. Often, one email criticizing minor design choices could derail the entire discussion. So, in 2021, I started a meetup where members of the typing community – some Python core developers (Guido van Rossum, Jelle Zijlstra), representatives from Mypy, Pyright, Pyre, etc. – all come together to discuss cutting-edge features. I’ve invited dozens of speakers on various topics and we’ve had some spirited but civil discussions. By having most of the stakeholders present, it has been easier to either get consensus or realize that something is not feasible in its current form. Some of the PEPs that have emerged after discussions are PEP 695 by Eric Traut for type variable syntax, PEP 692 for more precise **kwargs types by Franek Magiera (whom you have interviewed), and PEP 677 for nicer Callable syntax by Steven Troxler and me (though many of us were disappointed to see this one rejected :|).

What are your top three reasons for using Pyre over Mypy?

Ah, now we’re getting controversial! My personal reasons are:

Analysis beyond typechecking: Pyre is not just for typechecking. On top of Pyre, Meta has built a security-focused static analysis tool for Python, called Pysa, which is used extensively to detect security issues in Instagram’s code base. Pyre also has a powerful Query API that you can use to understand your code beyond just type errors: getting the function-call graph, class hierarchy, attributes, etc.Speed on large code bases: This was the key reason we built Pyre and why Instagram had switched from Mypy to Pyre back in 2018, though Mypy has made quite some strides in the meantime.First-class support for Meta’s infra: Overall, though, Pyre’s biggest strength is its support for Meta’s infra at scale – operating seamlessly with Meta’s internal build system (Buck), customizing features for a variety of projects, handling files with custom non-Python syntax, and powering code navigation. Scaling to such large, complex projects is where it really shines.

Please note that this is not meant to diminish Mypy’s genuine strengths in the open-source world. If you’re a smallish (< 5k LOC) open-source project thinking of adopting types, you really couldn’t go wrong using any of the major type checkers (Mypy, Pyre, Pyright, etc.).

Is there anything else you’d like to say?

If people are curious about the latest typing features in Python, want to get help adopting types in their projects, or want to contribute something back, please join the Python typing community on the Typing Discourse forum. We are a friendly community and welcome contributions. We also have a dedicated Typing Summit at PyCon each year (Jelle Zijlstra and I organized it last year), so if you are at PyCon just drop by.

Thanks so much for doing the interview, Pradeep!

The post PyDev of the Week: Pradeep Kumar Srinivasan appeared first on Mouse Vs Python.

PyDev of the Week: Sarah Boyce

Can you tell us a little about yourself (hobbies, education, etc):

Hello, I’m Sarah! I am a British software developer based in Germany.

I have enjoyed drawing portraits since I was little and nearly decided to study Art, but with some encouragement from my Dad, in the end I chose to study Mathematics. I still enjoy drawing and painting people when I can but I’m glad it’s a hobby and not my livelihood.

I’m also a badminton player! I play 2-3 times a week (or as often as I can) and I am part of a local club. We got promoted to a higher league this season and it’s tough! I’m pretty much always sore.

Otherwise, I really enjoy watching rugby. It’s the world cup at the minute and I’m enjoying all the games!

Why did you start using Python?

The first company I worked for used Django. In theory, I had done a little programming in my Maths degree, mostly MatLab and R, but Python was my introduction to professional programming.

I didn’t choose it but I’m glad it found me. Easy to read, great packages, strong community.

What other programming languages do you know and which is your favorite?

As a web developer, I also use JavaScript (mostly React), CSS and HTML. Then I use a few markup languages for various needs (YAML, MJML etc).

I think Python is my favourite but I might just be more proficient in it and better integrated into the Python community.

What projects are you working on now?

Mostly all things Django.

I am part of Django’s Review and Triage team (team of active contributors). So I work on tickets and engage in discussions around contributions to Django. I joined this team relatively recently, so there’s still lots to learn! I also write a “column” in the Django News weekly newsletter on the recent updates to Django which helps me stay very up-to-date with what’s going on.

I am a co-organiser of Djangonaut Space! This is a new mentorship program to onboard and develop Django contributors. We have nearly finished the pilot program and are really proud of what our Djangonauts have achieved in the 3 months they were with us. Now we are organising other events for the community, such as a Testathon for the new release of Django, and prepping for our next program (tentative planned start date for next January).

So there’s lots going on! But working with some really great people so it’s been a joy.

Which Python libraries are your favorite (core or 3rd party)?

Did I mention Django?

Why did you choose to participate in the Django project versus other Python web frameworks?

I didn’t consciously choose to work with Django, but it was a happy accident and I haven’t experienced any frustrations that had me reaching for other frameworks.

The Django community is also fantastic. I am really lucky there is a local meetup in my area and working on the framework is a joy.

I do want to try out FastAPI and Flask but I need to find something open source I can contribute to (if anyone has some projects I can maybe help with – I’m offering!) – I don’t want to just follow a tutorial, I want to solve problems.

What are your top three favorite features of Django?

These are not individual features but the ORM, migrations and the Django admin. These are such powerful features that I can’t imagine living without.

Is there anything else you’d like to say?

Djangonaut Space already has a lot of sign-ups for future programs. For us to support as many people as possible, we need more Captains and Navigators – people who can give Python mentorship and/or pastoral support. If that is something you would be interested in, please sign up.

Anyone who signs up will be invited to an info session – there’s no obligation from a sign-up, so don’t panic if you’re not 100% sure!

Thanks for doing the interview, Sarah!

The post PyDev of the Week: Sarah Boyce appeared first on Mouse Vs Python.

PyDev of the Week: Claudia Ng

This week we welcome Claudia Ng as the PyDev of the Week! Claudia is an author / contributor at Real Python! If you’d like to see what else Claudia has been up to, you should check out her personal website.

Let’s spend a few moments getting to know Claudia better!

Can you tell us a little about yourself (hobbies, education, etc):

I’m a data scientist and I’ve spent the past five years working in fraud and credit risk in the fintech (financial technology) space. I have a Masters in Public Policy from Harvard University and a Bachelor’s in International Business (Finance) with a minor in Spanish from Northeastern University.

In 2018, I was working at a Fintech called Tala, where I managed the new customer portfolio for their Mexico market. It was an incredible journey where we scaled the customer base by over 500x in only two years! Through this process I saw the power of automating lending decisions enabled by machine learning. I was fascinated by how alternative data could be used to predict customer’s repayment behaviors and fraud risk, unlocking the ability to lend to individuals with no or little credit history.

I’m an impact-driven person and seeing the power of applied ML inspired me to set my mind on pivoting into data science by taking on ML-related projects at work, doing online courses and side projects, and eventually moving onto the data science team.

I love what I do and outside of work, my hobbies include all kinds of water sports, bouldering and sudoku.

Why did you start using Python?

I first started using Python in 2019. I was initially using R for analyses since I had learned to use it in grad school, but the Data Science team used Python, so I started learning and picking it up. I found it to be more robust and there are many good third-party packages to support my work. Python is definitely my preferred language now!

What other programming languages do you know and which is your favorite?

I use Python and SQL daily on the job. I am a huge language nerd and can speak 9 human languages if that counts.

What projects are you working on now?

I am working on my second tutorial for Real Python on type hints for multiple return types in Python. Stay tuned for more when it comes out!

Which Python libraries are your favorite (core or 3rd party)?

I’m a Data Scientist, so I love pretty graphs and visuals. It is a crucial element to being able to tell a good data story and help with better decision-making. I would say that my favorite Python library is plotly. It’s a library for making interactive plots, and I love how versatile it is.

How did you get started writing articles for Real Python?

When I pivoted from an analyst role into data science back in 2019, I started writing because I wanted to share my learnings and hopefully inspire others without a STEM degree to break into data science/ engineering. I was writing blog posts on medium for several publications including Towards Data Science, Towards AI and Analytics Vidhya about different topics related to machine learning, feature engineering and data visualizations.

In early 2023, I saw that Real Python was looking for technical writers and applied. I was a subscriber and learned so much about programming from Real Python’s tutorials and courses, it feels like a dream to be writing for this publication!

What excites you most in the data science world right now?

I am excited about the rise of autoML packages that can automate some of the more tedious parts of ML modeling, like data cleaning, model selection and hyperparameter optimization. This would cut down the time spent during the model development cycle, allowing data scientists to iterate faster.

Is there anything else you’d like to say?

If you would like to check out my work, please visit ds-claudia.com to see past blog posts. You can also subscribe for free to receive emails when I publish new blog posts – no spam I promise!

Thanks so much for doing the interview, Claudia!

The post PyDev of the Week: Claudia Ng appeared first on Mouse Vs Python.

Learning About Code Metrics in Python with Radon

There are many different tools that you can install to help you be a better Python programmer. For example, you might install pytest so that you can do unit tests of your code. Or you might install Ruff, a super fast Python linter. The focus of this article is on another tool called Radon that helps you compute code metrics.

You can use Radon to help you find complex code in your code base. This is known as Cyclomatic Complexity or McCabe’s Complexity. According to Radon’s documentation:

“Cyclomatic Complexity corresponds to the number of decisions a block of code contains plus 1. This number (also called McCabe number) is equal to the number of linearly independent paths through the code. This number can be used as a guide when testing conditional logic in blocks.”

For example, if the number equals three, you will probably need to write at least three unit tests to have complete code coverage. Not only is this useful for figuring out how many tests to write, the cyclomatic complexity can tell you when it is time to refactor your code. You can read the full details of how Radon calculates complexity in their documentation.

Experienced developers are often able to know when to refactor from their own experience, but newer engineers may need the help that a tool like this provides.

You can also use a tool like Radon in your CI/CD system to prevent developers from merging in overly complex code.

Installing Radon

You can install Radon using pip. Here is an example:

python -m pip install radon

Now, let’s learn how to use Radon on your code!

Basic Radon Usage

You can call the radon application on the command line. Radon accepts multiple flags to control what kinds of output you receive.

Here are the four commands that radon currently can use:

cc: compute Cyclomatic Complexityraw: compute raw metricsmi: compute Maintainability Indexhal: compute Halstead complexity metrics

Let’s try running radon against the popular Black package (Black is a popular code formatted for Python).

Here is the command to run:

PS C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\black> radon cc . -a -nc
brackets.py
F 225:0 is_split_before_delimiter - F
M 70:4 BracketTracker.mark - C
comments.py
F 140:0 convert_one_fmt_off_pair - D
F 208:0 generate_ignored_nodes - C
F 253:0 _generate_ignored_nodes_from_fmt_skip - C
concurrency.py
F 120:0 schedule_formatting - C
files.py
F 309:0 gen_python_files - C
F 46:0 find_project_root - C
linegen.py
F 1133:0 normalize_invisible_parens - D
F 747:0 _maybe_split_omitting_optional_parens - D
F 1453:0 generate_trailers_to_omit - D
F 879:0 bracket_split_build_line - D
M 396:4 LineGenerator.visit_STRING - C
F 509:0 transform_line - C
F 997:0 delimiter_split - C
F 1529:0 run_transformer - C
F 1355:0 maybe_make_parens_invisible_in_atom - C
F 632:0 left_hand_split - C
M 289:4 LineGenerator.visit_simple_stmt - C
F 699:0 _first_right_hand_split - C
F 1313:0 remove_with_parens - C
lines.py
M 569:4 EmptyLineTracker._maybe_empty_lines - E
M 646:4 EmptyLineTracker._maybe_empty_lines_for_class_or_def - E
F 755:0 is_line_short_enough - D
C 514:0 EmptyLineTracker - D
F 882:0 can_omit_invisible_parens - C
M 300:4 Line.has_magic_trailing_comma - C
F 846:0 can_be_split - C
M 62:4 Line.append - C
M 529:4 EmptyLineTracker.maybe_empty_lines - C
M 228:4 Line.contains_uncollapsable_type_comments - C
M 362:4 Line.append_comment - C
nodes.py
F 174:0 whitespace - F
F 616:0 is_simple_decorator_trailer - C
F 573:0 is_one_sequence_between - C
parsing.py
F 164:0 stringify_ast - C
F 57:0 lib2to3_parse - C
strings.py
F 173:0 normalize_string_quotes - C
trans.py
M 792:4 StringParenStripper.do_match - D
M 1388:4 StringSplitter.do_transform - D
M 2070:4 StringParenWrapper.do_transform - D
M 1064:4 BaseStringSplitter._get_max_string_length - D
M 1334:4 StringSplitter.do_splitter_match - C
M 686:4 StringMerger._validate_msg - C
M 542:4 StringMerger._merge_one_string_group - C
F 1219:0 iter_fexpr_spans - C
C 772:0 StringParenStripper - C
M 378:4 StringMerger.do_match - C
C 1801:0 StringParenWrapper - C
M 1859:4 StringParenWrapper.do_splitter_match - C
F 84:0 hug_power_op - C
C 358:0 StringMerger - C
M 1174:4 BaseStringSplitter._prefer_paren_wrap_match - C
M 1985:4 StringParenWrapper._assign_match - C
M 2032:4 StringParenWrapper._dict_or_lambda_match - C
__init__.py
F 1156:0 get_features_used - F
F 443:0 main - E
F 616:0 get_sources - D
F 749:0 reformat_one - C
F 121:0 read_pyproject_toml - C
F 1094:0 _format_str_once - C
F 800:0 format_file_in_place - C

62 blocks (classes, functions, methods) analyzed.
Average complexity: C (19.741935483870968)

In this example, you asked radon to give you the Cyclomatic Complexity (cc) of the Black package. You also tacked on the -a or average flag and the -n flag, which lets you set the minimum complexity rank to display. The default is “a”, but in this example, you set the minimum to “c”, meaning it will show ranks C to F.

At this point, you can go through the code and start looking at the functions, methods, and classes to see what a C-ranked portion of code looks like compared with an F-ranked one. Give it a try and you’ll soon learn how to use radon to discover how complex your code is.

There are more command line options than what is shown here. Check out the full listing of additional flag in radon’s documentation.

You can also have radon measure the maintainability of your code by using the mi command. Here’s an example:

PS C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\black> radon mi .
brackets.py - A
cache.py - A
comments.py - A
concurrency.py - A
const.py - A
debug.py - A
files.py - A
handle_ipynb_magics.py - A
linegen.py - C
lines.py - C
mode.py - A
nodes.py - C
numerics.py - A
output.py - A
parsing.py - A
report.py - A
rusty.py - A
strings.py - A
trans.py - C
_width_table.py - A
__init__.py - C
__main__.py - A

The results of the mi command are similar to the cc command as radon will once again use letter ranks on your files to help you grade your code’s maintainability.

Radon can also calculate some raw metrics about your code by using the raw command. Here are the metrics that this command will calculate for you:

LOC: the total number of lines of codeLLOC: the number of logical lines of codeSLOC: the number of source lines of code – not necessarily corresponding to the LLOC [Wikipedia]comments: the number of Python comment lines (i.e. only single-line comments #)multi: the number of lines representing multi-line stringsblank: the number of blank lines (or whitespace-only ones)

Let’s try running the raw command against Black:

PS C:\Users\wheifrd\AppData\Local\Programs\Python\Python311\Lib\site-packages\black> radon raw .
brackets.py
LOC: 375
LLOC: 197
SLOC: 253
Comments: 4
Single comments: 11
Multi: 47
Blank: 64
- Comment Stats
(C % L): 1%
(C % S): 2%
(C M % L): 14%
cache.py
LOC: 97
LLOC: 59
SLOC: 54
Comments: 2
Single comments: 5
Multi: 14
Blank: 24
- Comment Stats
(C % L): 2%
(C % S): 4%
(C M % L): 16%
comments.py
LOC: 329
LLOC: 202
SLOC: 212
Comments: 33
Single comments: 29
Multi: 42
Blank: 46
- Comment Stats
(C % L): 10%
(C % S): 16%
(C M % L): 23%

The example above has been truncated since there is a LOT of output. But this snippet gives you a good idea of what to expect when you run the raw command against your code.

Wrapping Up

Radon is a great tool to add to your toolbox. You can use it to help you figure out which parts of your source code are getting too complex. You can also use Radon in your CI/CD infrastructure to prevent developers from checking in overly complex code. You might use this for your own open-source or commercial projects at your employer.

Give it a try and see if you like Radon!

The post Learning About Code Metrics in Python with Radon appeared first on Mouse Vs Python.

PyDev of the Week: Timothée Mazzucotelli

This week, we welcome Timothée Mazzucotelli (@pawamoy) as our PyDev of the Week! Timothée is the author of mkdocstrings.

You can see what else Timothée has been up to by visiting Timothée’s  showcase.

Let’s spend a few moments getting to know Timothée better!

Can you tell us a little about yourself (hobbies, education, etc):

My name is Timothée Mazzucoteli, @pawamoy on the web.

I love horror movies, horror stories, horror/survival games, really anything that relates to horrific stuff! I listen to “Nintendo Chill” as well as deathcore playlists. enjoy simple moments in life, like spending time with my significant other, walking around with my cats, and celebrating friendship around beers with my friends.

I discovered (X)HTML and CSS when I was around 15 years old, learning on a website called “Site du Zéro”. Then in high school I had the best math teacher ever, and thanks to him I went to learn Mathematics and Computer Science in college. Mathematics were *super hard*, but I got to enjoy Computer Science more and more, so for the second and third year I chose Comp.Sci. (you had to choose between the two). I discovered GNU/Linux (Ubuntu), OCaml, C, and many interesting parts of Computer Science, but mostly fell in love with the shell (Bash).

Why did you start using Python?

At the end of the third year in college, I asked a teacher if he had an apprenticeship for me (for the next two years), and he did. That’s how I started working with Python and Django, developing a website for a medical project called GenIDA. I worked on this project for four years. I was the only “IT” person, had no mentor, and a full-stack role: it was hard. But I learned a lot, and started publishing my first open source projects (Django apps) on GitHub.

What other programming languages do you know and which is your favorite?

My favorite language is Python, but I also love shell scripting. It doesn’t look like it, but you can do *a lot* with it. It’s my first love, and I miss it (full on Python these days). I was able to write libraries in sh/Bash/Zsh to add stack traces, profilers and debuggers to my toolbelt! These projects are on hold because I need to implement a `ZSH_XTRACE_FD` feature in Zsh (similar to the Bash equivalent) to improve debugging/profiling/coverage abilities, but Zsh uses a mailing list and I’m having a hard time feeling comfortable with this workflow :sweat_smile:

What projects are you working on now?

I’m mostly working within the MkDocs ecosystem, writing/maintaining plugins and Markdown extensions (mkdocstrings, Griffe, markdown-exec, etc.). When I need to refill my batteries I get some fun working on my own developer tools (git-changelog, duty, my project template copier-pdm), testing out new tools, or writing small posts on my website/blog.

Which Python libraries are your favorite (core or 3rd party)?

That has to be Jinja2 and Loguru. Loguru is so incredibly easy to use, and gives a beautiful output. Jinja2 is the base of many of my projects and ideas, I couldn’t do without it. I’m also super thankful for the `ast` module in the standard library. And HTTPX when I need to send network requests!

How did the mkdocstrings project come about?

Once upon a time, I wanted Sphinx to add entries in the table of contents for each auto-documented class, function, attribute, etc. It wasn’t possible, maintainers wouldn’t have the capacity to implement the feature before long, and I couldn’t wrap my head around Sphinx’ code base. At the same time, FastAPI got super popular, and when I saw its documentation pages, as well as Pydantic’s, I thought: “This is pretty, I want that.” It was not the first time I had seen MkDocs and the amazing Material for MkDocs theme, but this time it clicked. I found mkautodoc, by @tomchristie, but Tom didn’t have a lot of time to spent on it either, reviewing PRs or implementing features, so I decided I would create my own “autodoc” plugin for MkDocs.

What challenges do you face as an author of a popular open-source package?

Maintaining/developing popular open-source projects is a tremendous amount of work, and often I miss out on other things I could do in life. I enjoy open source a lot, but sometimes it manifests more as an obsession rather than a passion. It can easily become unhealthy. It’s hard to find the balance sometimes.

But the biggest challenge is happening right now: I quit my job to work on open-source full-time. I couldn’t stand working for others anymore, on projects that don’t really interest me. So I’m taking a bet that the community will be here for me and my projects, funding me so that I can actually get the time to work on these projects (and also get a decent life).

Is there anything else you’d like to say?

The Python ecosystem and community are amazing, and I’m very thankful to be part of it

Thanks so much for doing the interview, Timothée!

The post PyDev of the Week: Timothée Mazzucotelli appeared first on Mouse Vs Python.

PyDev of the Week: Jelle Zijlstra

This week, we welcome Jelle Zijlstra as our PyDev of the Week! Jelle is a core developer of the Python programming language. If you’d like to see what exactly Jelle is working on, you should check out his GitHub profile, Quora, or the Python Packaging Index.

Let’s take a few moments to get to know Jelle better!

Can you tell us a little about yourself (hobbies, education, etc):

I grew up in the Netherlands wanting to be a biologist, but decided to go to the United States for college. Then I took a computer science class, took a few more, and decided that software engineering offered better career prospects than biology. I still graduated with a degree in biology, but found a job as a software engineer.

Still, biology remains one of my main hobbies: I maintain a big database of biological nomenclature at https://hesperomys.com. I started out doing this in spreadsheets, and now it is a SQLite database. Learning to program has been hugely valuable for building up a high-quality, interconnected database.

Before the pandemic I used to walk much of the way to work, but then I was stuck at home, so I decided to go on walks anyway to exercise. That quickly spiraled into ridiculously long walks around the San Francisco Bay Area, up to 57 miles in a day. I write about my walks at https://walkingbayarea.quora.com/.

But the reason you’re interviewing me is my open source work. At my job a few years ago I got to start a new service that was going to be in Python 3, so I got to use shiny new tools like typing and asyncio. I ran into some minor issues with typing, started contributing to the relevant open source projects, and got sucked in. Now I help maintain several important parts of the Python ecosystem, including typing-extensions, typeshed, Black, mypy, and CPython.

Why did you start using Python?

The first language I learned in college was C. I’m still not sure whether I think C is a good first language to teach to students, but it’s definitely been valuable in giving me an understanding of how basic concepts such as pointers and memory work. Later in that class, we did a little bit of web programming in PHP. PHP isn’t a great language, but for my personal programming projects (such as maintaining my mammal database), it was still a better fit than C—no segfaults and much more library support. But I had to use Python in another class, and I quickly realized it was a lot better than PHP. Then I started a job mostly using Python, so it remained my main language.

What other programming languages do you know and which is your favorite?

When I interviewed for my current job, during each interview I picked the language that felt like the best fit for the question, and at the end of the day it turned out I had used a different language in each interview! I think it was C, Python, JavaScript, and OCaml. More recently, the main non-Python languages I’ve used have been TypeScript and C++, plus a bit of Swift. I think languages are valuable when they teach you something new about programming. OCaml was the most valuable language I learned in college because it taught me a completely new style of programming. Among languages I have dabbled in since, Haskell and Rust have been most useful.

What projects are you working on now?

My most recent major open-source contribution has been implementing PEP 695 for Python 3.12, which is coming out in October. It’s a major syntactic change that makes it a lot easier to write generics in Python. I wrote up a detailed account of the implementation at https://jellezijlstra.github.io/pep695.

At my job, I am now working on Poe, an AI chat app. Outside of work, I’ve been focusing recently on my mammal database instead of open-source software.

Which Python libraries are your favorite (core or 3rd party)?

typing. It’s been an invaluable tool to keep our huge Python codebase at work manageable.

How did you get into doing core Python development?

I attended a CPython sprint at PyCon in Portland and Cleveland before the pandemic and contributed a few things; for example, you can thank me for `@contextlib.asynccontextmanager`. However, the CPython core always felt a little remote and hard to get into, with the unusual workflow at the time (e.g., using Gerrit) and long release cycle. But then one day, Guido van Rossum emailed me to ask whether I was interested in becoming a core dev, so I said yes and after spending a few months becoming more familiar with the workflow (which is now a lot less unusual), I was voted in as a core dev. Guido asked me because at the time he was basically maintaining typing.py by himself, and obviously he is very busy. Now, we have several other people helping out in that area.

What are some new features in Python that you’re excited about?

I’m excited about the new syntax for generics and type aliases that I helped get into Python 3.12. Longer term, Python 3.13 should ship with deferred evaluation of annotations (PEP 649), which will make a lot of code that uses type annotations more ergonomic. We’re also likely to ship support for inline TypedDict types, another nice usability improvement.

Thanks so much for doing the interview, Jelle!

The post PyDev of the Week: Jelle Zijlstra appeared first on Mouse Vs Python.

How to Validate an IP Address in Python

Validating data is one of the many tasks engineers face around the world. Your users make typos as often as anyone else, so you must try to catch those errors.

One common type of information that an engineer will need to capture is someone who wants to edit or enter an IP address for a device. These addresses are made up of four sets of numbers separated by periods. Each of these four sets are made up of integers from 0-255.

The lowest IP address would be 0.0.0.0, and the highest would be 255.255.255.255.

In this article, you will look at the following ways to validate an IP address with Python:

Using the socket moduleUsing the ipaddress module

Let’s get started!

Using the socket Module

Python has lots of modules or libraries built-in to the language. One of those modules is the socket module.

Here is how you can validate an IP address using the socket module:

import socket

try:
socket.inet_aton(ip_address)
print(f"{ip_address is valid")
except socket.error:
print(f"{ip_address} is not valid!")

Unfortunately, this doesn’t work with all valid IP addresses. For example, this code won’t work IPv6 variants, only IPv4. You can use socket.inet_pton(socket_family, address)  and specify which IP version you want to use, which may work. However, the inet_pton() function is only available on Unix.

Luckily, there is a better way!

Using the ipaddress Module

Python also includes the ipaddress module in its standard library. This module is available for use on all platforms and can be used to validate IPv4 and IPv6.

Here is how you can validate an IP address using Python’s built-in ipaddress module:

import ipaddress

def ip_address_validator(ip):
try:
ip_obj = ipaddress.ip_address(ip)
print(f"{ip} is a valid IP address")
except ValueError:
print(f"ERROR: {ip} is not a valid IP address!")

ip_address_validator("192.168.5.1")

The ipaddress module makes validating IP addresses even simpler than the socket module!

Wrapping Up

The Python programming language provides a couple of modules that you can use to validate IP addresses. You can use either the socket module or the ipaddress module. The socket module has some limitations on platforms other than Unix. That makes using the ipaddress module preferable since you may use it on any platform.

Note: You could probably find a regular expression that you could use to validate an IP address, too. However, leveraging the ipaddress module is simpler and easier to debug.

The post How to Validate an IP Address in Python appeared first on Mouse Vs Python.