Document current state and guidance for beginner installs

[matthew-brett]

It is a vexed question as to which installation method we should recommend for complete beginners, with various trade-offs and issues. For example, see discussion in Scipy.org PR installation page, where the current state of that page is: Scipy beginner installation guide.

It would be very useful to set out the issues that one might consider in recommending a user install method, so the projects can either defer to a central page - perhaps based at scientific-python, or modify their advice to their own taste.

At a first pass, here are some issues that would benefit from wise and experienced reflection:

• There are different types of beginners. Some beginners already know their way around computers; they know how to edit configuration files, and they have a fluent understanding of filesystems, and the use of the terminal. Many beginners - such as absolute beginners in a data science or statistics class - do not have such knowledge.
• In general, many installation pages target relatively experienced computer users.
• Most of us writing such pages are extremely experienced computer users.
• We experts tend to see more experienced or committed beginners.
• Some tools and workflows that seem natural and desirable to experienced users are major barriers to beginners. Virtualenvs are one example - but an important one, as pip has increasingly tried to force users to use virtualenvs, and uv has a similar approach.
• The change in Anaconda's terms of use have made Anaconda installs more ambiguous as a default method for beginners.
• Pip's move to force users to use virtualenvs has made system installs of Python considerably less friendly to beginners - for example via Homebrew - see https://github.com/orgs/Homebrew/discussions/3404 .
• Some installation tools have installation methods that also need some experience to understand and use. I would argue Mamba is one such example.

So - it would be useful to take some time to lay out the options, note the tradeoffs, and set out the sources of confusion and difficulty that true beginners may have. We could then follow up with a standard set of guideliness, perhaps with different flavors for those more or less tolerant of the various downsides that we identify. We might also work with the installation tool authors to find ways of making it easier for true beginners to install the installation tools.

[jarrodmillman]

https://learn.scientific-python.org/contributors/setup/install/

[dschult]

It would be nice to learn from R in regards to this question. When I have taught a class using R, the students have much less trouble getting started than when using Python. The whole install/import process seems much cleaner in R -- and of course, RStudio and CRAN make it easier too. I don't know how CRAN handles dependency versions, but from a user perspective, it is very smooth.

[matthew-brett]

@dschult - yes - that's a good comparison. My advice for R has been to follow the instructions on the RStudio download page, which boil down to:

• Download R installer from R website and double-click install
• Download RStudio installer from Posit website and double-click install.

Is that the kind of advice you have given?

Of course, with R, you get most of what you need for an absolute beginner class by installing base R. I have personally run into some nasty troubles installing or reinstalling complex packages from CRAN - probably more so than for Python / Pip.

Three of our problems with Python installation are that:

• Python has greater representation from computer science, and therefore, a higher tolerance for instructions that will be overwhelming for true beginners, and
• There are so many different ways to install Python and
• The double click installation method most widely used was Anaconda, now less attractive because of the change in the terms of use.

I have in the past built one-click installers for the Scipy-stack on Mac : https://fullmonty.github.io - I guess it would only take a day or so to get those back into shape, and I guess it would only take a week or so to make a first pass at something similar for Windows.

Maybe we should consider that as part of the discussion.

[matthewfeickert]

The whole install/import process seems much cleaner in R -- and of course, RStudio and CRAN make it easier too. I don't know how CRAN handles dependency versions, but from a user perspective, it is very smooth.

I would disagree with this when it comes to Linux. Yes, for Windows and macOS users the CRAN experience is pretty good, but for any Linux users CRAN is essentially the equivalent of sdists only and there is essentially no coordinated effort to improve this AFAIK (there is Posit's package management system, but that both requires users to know that it exists, be able to navigate the (to me) very confusingly laid out docs, and understand that they need to run fully different scripts to install from it. Same with R-Universe.). I ran into this when working on contributing lessons to The Carpentries, which has gone all in on RMarkdown for developing lessons. For someone like me this was not that big of a pain, but when I realized that I was going to have to build 40+ packages from source and a non-zero number of them failed in my Pixi workspace given they assumed the existence of developer libraries I realized that for beginners this would be terrible.

So this is just echoing @matthew-brett's observation that

I have personally run into some nasty troubles installing or reinstalling complex packages from CRAN - probably more so than for Python / Pip.

Luckily, for me and for others, conda-forge is filling in as the missing link for R users as it has very good (multi-platform) support for most common R-package (and I am now maintaining a few others just to make my lesson dev better carpentries/workbench#90).

My point in mentioning all of this is that I really do not think that R is a good example of how to do onboarding well for everyone, and there are people (who I won't publicly bring in here) in Posit who I have talked to and agree with me on this.

Where I fully agree with @dschult though is that RStudio has successfully become a (pretty great) one stop shop for all R installation and development needs. Though from talking with people who are actual R developers, they seem to also attribute this to the fact that most R users do not care about having partitioned (potentially transitory) R environments and that they are content with having in most cases one (nearly) global R environment in which everything gets installed. There is also no official support for conda environments in RStudio and no plan to do so AFAIK from Posit, so for people who start in RStudio the very good R environment control that conda-forge conda packages offer is out of reach.

@matthew-brett @dschult I would in general be interested in discussing with you this week what your view on the proper level of semantics are for beginners. I'm very biased in my views, but I've recently started to think that Pixi (and to some extent uv) have a viable level of semantics that don't require a new users to have to think or care much about the fact that they're interacting with a virtual environment. With Pixi projects new users don't even need to know that they installed anything (if they don't want to).

[drammock]

most R users do not care about having partitioned (potentially transitory) R environments and that they are content with having in most cases one (nearly) global R environment in which everything gets installed.

this fits with my (somewhat outdated) impression of R users when I regularly walked among them. But it doesn't have to be that way: in addition to conda-forge's R support that @matthewfeickert mentioned, renv has been around for several years now I think. But I don't have a clear sense of what it's uptake is like, or what reasons folks might have to not use it.

[dschult]

My advice for R has been to follow the instructions on the RStudio download page, which boil down to:

Download R installer from R website and double-click install
Download RStudio installer from Posit website and double-click install.
Is that the kind of advice you have given?

Yes -- those were the steps that I gave the students and it went pretty well. Of course they weren't worried about versioned libraries. :)

And everyone is right to point out that the CRAN system is not very friendly to versioning. But that's not even on the radar for most beginners -- especially students. The CRAN system reminds me of the CTAN/LaTeX model of software development where instead of a new version you just make a new package, and it seems like no release is ever outdated. That has advantages (old code still works) and disadvantages (hard to update things). And it is a generalization of what really happens (there are new versions) but it is a short perhaps pithy way to describe that style of code evolution. I don't think the Python language community wants that style.

It would be nice for beginners to have an onramp that doesn't require them to fuss with environments, library installs, versions and command lines (other than the interpreter) and even choice of editors. I think the Python tools are close to this. Anaconda almost made it (my students were overwhelmed with choices when using that system). pip and jupyter are improved and close to R's "install and load from a single app" approach. But I haven't looked at these kinds of options in a few years. New tools like pixi and uv may solve many issues idk.

[matthewfeickert]

But I haven't looked at these kinds of options in a few years. New tools like pixi and uv may solve many issues idk.

I can be somewhat accurately called a Pixi evangelist, but Pixi does not get you away form the command line (I'm not really sure if that's something that we really need to solve) but for most situations the only thing any student needs to type is

pixi run start

and that's it. Example (I have others if of interest): https://github.com/matthewfeickert/jupytergis-neon-open-data-exploration

[matthew-brett]

I have views! But should we gather soon to hash this out?

[matthew-brett]

For those of us in the meeting - we will meet in main room at the coffee machines at 4.45 today (Sunday) to find a room for 30 minutes of range-finding on beginner install issues. Anyone welcome.

[matthew-brett]

To summarize:

• True beginners may well not understand the file-system hierarchy.

• Lack of fluency with the file-system can make it difficult for these beginner to use the terminal to navigate to the directory containing a file they have downloaded.

• It is no great barrier to ask true beginners to open the terminal and run some initial commands.

• Anaconda was a good solution before the terms-of-use-change, because it provided a familiar double-click install that, in turn added Start menu (on Windows) options to start configured terminals, or the Jupyter notebook - so the student can get started at once with a new notebook.

• @drammock pointed out that MNE has similar double-click installers, available at https://mne.tools/stable/install/installers.html, and built from https://github.com/mne-tools/mne-installers. These are based on Conda and draw exclusively from conda-forge, therefore avoiding the Anaconda terms-of-use. We plan to take these off to another repo to generalize them to allow configuration of installed packages, logos etc.

• Tools like uv and pixi depend on the user having an idea of projects where projects can be defined by configuration files in directories, and this can be a barrier for beginner users.

• It is generally advisable to minimize the amount of just-trust-me mechanics in classes, because this tends to induce a state of confusion and insecurity in the students, especially if they accidentally follow some new path and run into errors they don't understand, such as:

  $ pixi run start
  Error:   × could not find pixi.toml or pyproject.toml at directory /Users/mb312/Documents
  

• Nevertheless, students who will likely go on to need virtual environments in their work may benefit from early use of these tools.

• We need to consider when to introduce ways of installing their own environments using more standard (developer) installation methods, rather than relying on double-click installers. Different courses will introduce these methods at different times.

[lucascolley]

I agree that https://scipy.org/beginner-install/ is not in a great place. I am quite happy with the state of https://scipy.org/install/ now for intermediate users, but there is a big gap that we are not covering all so well.

---

+1 for GUI installers, I think they are very beginner friendly! Perhaps https://github.com/Quantco/pixi-pack can one day grow support for something like https://github.com/conda/constructor's installers.

Tools like uv and pixi depend on the user having an idea of projects where projects can be defined by configuration files in directories, and this can be a barrier for beginner users.

I suppose so, although for just pixi run start, it is more the idea of directories which beginners need to get a grasp of (as pointed out above), not projects per se.

Error: × could not find pixi.toml or pyproject.toml at directory /Users/mb312/Documents

Perhaps we could consider adding some generic "you may want to check you are in the directory of the project you are trying to use" here, although admittedly that does suggest that users should at least understand what the word 'project' roughly means.

I think the bigger problem for uv and pixi to be the way forward for beginners (which I hope they will be one day) is @stefanv's point about tutorial materials, online guides etc. referring to (usually just) global pip installations. A paradigm shift is needed to get away from that, but such a shift is really difficult to execute when you have new tools vs decades of materials that a user might stumble across. I don't really know how best to proceed here, other than disparately fighting the war of attrition. Coordination across the Scientific Python ecosystem will be a big help, but I guess a lot more may be needed in the grand scheme of things.

[matthew-brett]

Just to clarify - I think it's important to distinguish between beginners at coding, and beginner developers.

I could certainly see the argument for pressing beginner developers to use virtualenvs or project-based-installs early, although, personally, I would not do that - I would introduce these tools relatively late.

However, the users we are talking about here are beginners at coding. For many of these users, it will be many years until they reach the stage where they will get substantial benefit from virtualenvs / project-based installs. I've argued above that requiring these users to navigate the various traps involved in these types of installs is a significant requirement - and therefore, the cost is too high compared to the - initially - negligible benefit.

So - if it were me - I would continue to advocate that beginner coder installs do not press users towards virtualenv / project-based installs. I continue to have a strong preference for "Double click install, start Jupyter (doesn't matter what directory you are in), then import pandas as pd" etc. We may well have a suitable installer built by the end of this week.

In terms of instructions, I think this boils down to something like:

"If you just want to get started, double click this installer and ... If you think you might want to get involved in development, consider using (uv, pixi ...), see instructions below."

[rootsmusic]

I think it's important to distinguish between beginners at coding, and beginner developers.

The Development Guide uses pip or conda to create a development environment. Since Scientific Python Lectures are intended for coding beginners, see issue to update the section to install a working environment.