.. _upgrading: ================== Upgrading from 1.x ================== Modern Fabric (2+) represents a near-total reimplementation & reorganization of the software. It's been :ref:`broken in two `, cleaned up, made more explicit, and so forth. In some cases, upgrading requires only basic search & replace; in others, more work is needed. If you read this document carefully, it should guide you in the right direction until you're fully upgraded. If any functionality you're using in Fabric 1 isn't listed here, please file a ticket `on Github `_ and we'll update it ASAP. .. warning:: As of the 2.0 release line, Fabric 2 is **not** at 100% feature parity with 1.x! Some features have been explicitly dropped, but others simply have not been ported over yet, either due to time constraints or because said features need to be re-examined in a modern context. Please review the information below, including the :ref:`upgrade-specifics` section which contains a very detailed list, before filing bug reports! Also see :ref:`the roadmap ` for additional notes about release versioning. Why upgrade? ============ We'd like to call out, in no particular order, some specific improvements in modern Fabric that might make upgrading worth your time. .. note:: These are all listed in the rest of the doc too, so if you're already sold, just skip there. - Thread-safe - no more requirement on multiprocessing for concurrency; - API reorganized around `fabric.connection.Connection` objects instead of global module state; - Command-line parser overhauled to allow for regular GNU/POSIX style flags and options on a per-task basis (no more ``fab mytask:weird=custom,arg=format``); - Task organization is more explicit and flexible / has less 'magic'; - Tasks can declare other tasks to always be run before or after themselves; - Configuration massively expanded to allow for multiple config files & formats, env vars, per-user/project/module configs, and much more; - SSH config file loading enabled by default & has been fleshed out re: system/user/runtime file selection; - Shell command execution API consistent across local and remote method calls - no more differentiation between ``local`` and ``run`` (besides where the command runs, of course!); - Shell commands significantly more flexible re: interactive behavior, simultaneous capture & display (now applies to local subprocesses, not just remote), encoding control, and auto-responding; - Use of Paramiko's APIs for the SSH layer much more transparent - e.g. `fabric.connection.Connection` allows control over the kwargs given to `SSHClient.connect `; - Gateway/jump-host functionality offers a ``ProxyJump`` style 'native' (no proxy-command subprocesses) option, which can be nested infinitely; 'Sidegrading' to Invoke ======================= We linked to a note about this above, but to be explicit: modern Fabric is really a few separate libraries, and anything not strictly SSH or network related has been :ref:`split out into the Invoke project `. This means that if you're in the group of users leveraging Fabric solely for its task execution or ``local``, and never used ``run``, ``put`` or similar - **you don't need to use Fabric itself anymore** and can simply **'sidegrade' to Invoke instead**. You'll still want to read over this document to get a sense of how things have changed, but be aware that you can get away with ``pip install invoke`` and won't need Fabric, Paramiko, cryptography dependencies, or anything else. Using modern Fabric from within Invoke ====================================== We intend to enhance modern Fabric until it encompasses the bulk of Fabric 1's use cases, such that you can use ``fab`` and fabfiles on their own without caring too much about how it's built on top of Invoke. However, prior to that point -- and very useful on its own for intermediate-to-advanced users -- is the fact that modern Fabric is designed with library or direct API use in mind. **It's entirely possible, and in some cases preferable, to use Invoke for your CLI needs and Fabric as a pure API within your Invoke tasks.** In other words, you can eschew ``fab``/fabfiles entirely unless you find yourself strongly needing the conveniences it wraps around ad-hoc sessions, such as :option:`--hosts` and the like. Running both Fabric versions simultaneously =========================================== To help with gradual upgrades, modern Fabric may be installed under the name ``fabric2`` (in addition to being made available "normally" as versions 2.0+ of ``fabric``) and can live alongside installations of version 1.x. Thus, if you have a large codebase and don't want to make the jump to modern versions in one leap, it's possible to have both Fabric 1 (``fabric``, as you presumably had it installed previously) and modern Fabric (as ``fabric2``) resident in your Python environment simultaneously. .. note:: We strongly recommend that you eventually migrate all code using Fabric 1, to versions 2 or above, so that you can move back to installing and importing under the ``fabric`` name. ``fabric2`` as a distinct package and module is intended to be a stopgap, and there will not be any ``fabric3`` or above (not least because some of those names are already taken!) For details on how to obtain the ``fabric2`` version of the package, see :ref:`installing-as-fabric2`. .. _from-v1: Creating ``Connection`` and/or ``Config`` objects from v1 settings ------------------------------------------------------------------ A common tactic when upgrading piecemeal is to generate modern Fabric objects whose contents match the current Fabric 1 environment. Whereas Fabric 1 stores *all* configuration (including the "current host") in a single place -- the ``env`` object -- modern Fabric breaks things up into multiple (albeit composed) objects: `~fabric.connection.Connection` for per-connection parameters, and `~fabric.config.Config` for general settings and defaults. In most cases, you'll only need to generate a `~fabric.connection.Connection` object using the alternate class constructor `Connection.from_v1 `, which should be fed your appropriate local ``fabric.api.env`` object; see its API docs for details. A contrived example:: from fabric.api import env, run from fabric2 import Connection env.host_string = "admin@myserver" run("whoami") # v1 cxn = Connection.from_v1(env) cxn.run("whoami") # v2+ By default, this constructor calls another API member -- `Config.from_v1 ` -- internally on your behalf. Users who need tighter control over modern-style config options may opt to call that classmethod explicitly and hand their modified result into `Connection.from_v1 `, which will cause the latter to skip any implicit config creation. .. _v1-env-var-imports: Mapping of v1 ``env`` vars to modern API members ------------------------------------------------ The ``env`` vars and how they map to `~fabric.connection.Connection` arguments or `~fabric.config.Config` values (when fed into the ``.from_v1`` constructors described above) are listed below. .. list-table:: :header-rows: 1 * - v1 ``env`` var - v2+ usage (prefixed with the class it ends up in) * - ``always_use_pty`` - Config: ``run.pty``. * - ``command_timeout`` - Config: ``timeouts.command``; timeouts are now their own config subtree, whereas in v1 it was possible for the ambiguous ``timeout`` setting -- see below -- to work for either connect OR command timeouts. * - ``forward_agent`` - Config: ``connect_kwargs.forward_agent``. * - ``gateway`` - Config: ``gateway``. * - ``host_string`` - Connection: ``host`` kwarg (which can handle host-string like values, including user/port). * - ``key`` - **Not supported**: Fabric 1 performed extra processing on this (trying a bunch of key classes to instantiate) before handing it into Paramiko; modern Fabric prefers to just let you handle Paramiko-level parameters directly. If you're filling your Fabric 1 ``key`` data from a file, we recommend switching to ``key_filename`` instead, which is supported. If you're loading key data from some other source as a string, you should know what type of key your data is and manually instantiate it instead, then supply it to the ``connect_kwargs`` parameter. For example:: from io import StringIO from fabric.state import env from fabric2 import Connection from paramiko import RSAKey from somewhere import load_my_key_string pkey = RSAKey.from_private_key(StringIO(load_my_key_string())) cxn = Connection.from_v1(env, connect_kwargs={"pkey": pkey}) * - ``key_filename`` - Config: ``connect_kwargs.key_filename``. * - ``no_agent`` - Config: ``connect_kwargs.allow_agent`` (inverted). * - ``password`` - Config: ``connect_kwargs.password``, as well as ``sudo.password`` **if and only if** the env's ``sudo_password`` (see below) is unset. (This mimics how v1 uses this particular setting - in earlier versions there was no ``sudo_password`` at all.) * - ``port`` - Connection: ``port`` kwarg. Is casted to an integer due to Fabric 1's default being a string value (which is not valid in v2). .. note:: Since v1's ``port`` is used both for a default *and* to store the current connection state, v2 uses it to fill in the Connection only, and not the Config, on assumption that it will typically be the current connection state. * - ``ssh_config_path`` - Config: ``ssh_config_path``. * - ``sudo_password`` - Config: ``sudo.password``. * - ``sudo_prompt`` - Config: ``sudo.prompt``. * - ``timeout`` - Config: ``timeouts.connection``, for connection timeouts, or ``timeouts.command`` for command timeouts (see above). * - ``use_ssh_config`` - Config: ``load_ssh_configs``. * - ``user`` - Connection: ``user`` kwarg. * - ``warn_only`` - Config: ``run.warn`` .. _upgrade-specifics: Upgrade specifics ================= This is (intended to be) an exhaustive list of *all* Fabric 1.x functionality, as well as new-to-Invoke-or-Fabric-2 functionality not present in 1.x; it specifies whether upgrading is necessary, how to upgrade if so, and tracks features which haven't been implemented in modern versions yet. Most sections are broken down in table form, as follows: .. list-table:: * - Fabric 1 feature or behavior - Status, see below for breakdown - Migration notes, removal rationale, etc Below are the typical values for the 'status' column, though some of them are a bit loose - make sure to read the notes column in all cases! Also note that things are not ironclad - eg any 'removed' item has some chance of returning if enough users request it or use cases are made that workarounds are insufficient. - **Ported**: available already, possibly renamed or moved (frequently, moved into the `Invoke `_ codebase.) - **Pending**: would fit, but has not yet been ported, good candidate for a patch. *These entries link to the appropriate Github ticket* - please do not make new ones! - **Removed**: explicitly *not* ported (no longer fits with vision, had too poor a maintenance-to-value ratio, etc) and unlikely to be reinstated. Here's a quick local table of contents for navigation purposes: .. contents:: :local: .. _upgrading-general: General / conceptual -------------------- - The CLI task-oriented workflow remains a primary design goal, but the library use case is no longer a second-class citizen; instead, the library functionality has been designed first, with the CLI/task features built on top of it. - Additionally, within the CLI use case, version 1 placed too much emphasis on 'lazy' interactive prompts for authentication secrets or even connection parameters, driven in part by a lack of strong configuration mechanisms. Over time it became clear this wasn't worth the tradeoffs of having confusing noninteractive behavior and difficult debugging/testing procedures. Modern Fabric takes an arguably cleaner approach (based on functionality added to v1 over time) where users are encouraged to leverage the configuration system and/or serve the user prompts for runtime secrets at the *start* of the process; if the system determines it's missing information partway through, it raises exceptions instead of prompting. - Invoke's design includes :ref:`explicit user-facing testing functionality `; if you didn't find a way to write tests for your Fabric-using code before, it should be much easier now. - We recommend trying to write tests early on; they will help clarify the upgrade process for you & also make the process safer! .. _upgrading-api: API organization ---------------- High level code flow and API member concerns. .. list-table:: :widths: 40 10 50 * - Import everything via ``fabric.api`` - Removed - All useful imports are now available at the top level, e.g. ``from fabric import Connection``. * - Configure connection parameters globally (via ``env.host_string``, ``env.host``, ``env.port``, ``env.user``) and call global methods which implicitly reference them (``run``/``sudo``/etc) - Removed - The primary API is now properly OOP: instantiate `fabric.connection.Connection` objects and call their methods. These objects encapsulate all connection state (user, host, gateway, etc) and have their own SSH client instances. .. seealso:: `Connection.from_v1 ` * - Emphasis on serialized "host strings" as method of setting user, host, port, etc - Ported/Removed - `fabric.connection.Connection` *can* accept a shorthand "host string"-like argument, but the primary API is now explicit user, host, port, etc keyword arguments. Additionally, many arguments/settings/etc that expected a host string in v1 will now expect a `fabric.connection.Connection` instance instead. * - Use of "roles" as global named lists of host strings - Ported - This need is now served by `fabric.group.Group` objects (which wrap some number of `fabric.connection.Connection` instances with "do a thing to all members" methods.) Users can create & organize these any way they want. See the line items for ``--roles`` (:ref:`upgrading-cli`), ``env.roles`` (:ref:`upgrading-env`) and ``@roles`` (:ref:`upgrading-tasks`) for the status of those specifics. .. _upgrading-tasks: Task functions & decorators --------------------------- .. note:: Nearly all task-related functionality is implemented in Invoke; for more details see its :ref:`execution ` and :ref:`namespaces ` documentation. .. list-table:: :widths: 40 10 50 * - By default, tasks are loaded from a ``fabfile.py`` which is sought up towards filesystem root from the user's current working directory - Ported - This behavior is basically identical today, with minor modifications and enhancements (such as tighter control over the load process, and API hooks for implementing custom loader logic - see :ref:`loading-collections`.) * - "Classic" style implicit task functions lacking a ``@task`` decorator - Removed - These were on the way out even in v1, and arbitrary task/namespace creation is more explicitly documented now, via Invoke's `~invoke.tasks.Task` and `~invoke.collection.Collection`. * - "New" style ``@task``-decorated, module-level task functions - Ported - Largely the same, though now with superpowers - `@task ` can still be used without any parentheses, but where v1 only had a single ``task_class`` argument, the new version (largely based on Invoke's) has a number of namespace and parser hints, as well as execution related options (such as those formerly served by ``@hosts`` and friends). * - Arbitrary task function arguments (i.e. ``def mytask(any, thing, at, all)``) - Ported - This gets its own line item because: tasks must now take a `~invoke.context.Context` (vanilla Invoke) or `fabric.connection.Connection` (Fabric) object as their first positional argument. The rest of the function signature is, as before, totally up to the user & will get automatically turned into CLI flags. This sacrifices a small bit of the "quick DSL" of v1 in exchange for a cleaner, easier to understand/debug, and more user-overrideable API structure. As a side effect, it lessens the distinction between "module of functions" and "class of methods"; users can more easily start with the former and migrate to the latter when their needs grow/change. * - Implicit task tree generation via import-crawling - Ported/Removed - Namespace construction is now more explicit; for example, imported modules in your ``fabfile.py`` are no longer auto-scanned and auto-added to the task tree. However, the root ``fabfile.py`` *is* automatically loaded (using `Collection.from_module `), preserving the simple/common case. See :ref:`task-namespaces` for details. We may reinstate (in an opt-in fashion) imported module scanning later, since the use of explicit namespace objects still allows users control over the tree that results. * - ``@hosts`` for determining the default host or list of hosts a given task uses - Ported - Reinstated as the ``hosts`` parameter of `@task `. Further, it can now handle dicts of `fabric.connection.Connection` kwargs in addition to simple host strings. * - ``@roles`` for determining the default list of group-of-host targets a given task uses - Pending - See :ref:`upgrading-api` for details on the overall 'roles' concept. When it returns, this will probably follow ``@hosts`` and become some ``@task`` argument. * - ``@serial``/``@parallel``/``@runs_once`` - Ported/`Pending `__ - Parallel execution is currently offered at the API level via `fabric.group.Group` subclasses such as `fabric.group.ThreadingGroup`; however, designating entire sessions and/or tasks to run in parallel (or to exempt from parallelism) has not been solved yet. The problem needs solving at a higher level than just SSH targets, so this links to an Invoke-level ticket. * - ``execute`` for calling named tasks from other tasks while honoring decorators and other execution mechanics (as opposed to calling them simply as functions) - `Pending `__ - This is one of the top "missing features" from the rewrite; link is to Invoke's tracker. * - ``Task`` class for programmatic creation of tasks (as opposed to using some function object and the ``@task`` decorator) - Ported - While not sharing many implementation details with v1, modern Fabric (via Invoke) has a publicly exposed `~invoke.tasks.Task` class, which alongside `~invoke.collection.Collection` allow full programmatic creation of task trees, no decorator needed. .. _upgrading-cli: CLI arguments, options and behavior ----------------------------------- .. list-table:: :widths: 40 10 50 * - Exposure of task arguments as custom colon/comma delimited CLI arguments, e.g. ``fab mytask:posarg,kwarg=val`` - Removed - CLI arguments are now proper GNU/POSIX-style long and short flags, including globbing shortflags together, space or equals signs to attach values, optional values, and much more. See :ref:`invoking-tasks`. * - Task definition names are mirrored directly on the command-line, e.g for task ``def journald_logs()``, command line argument is ``fab journald_logs`` - Removed - Tasks names now get converted from underscores to hyphens. Eg. task ``def journald_logs()`` now evaluates to ``fab journald-logs`` on the commandline. * - Ability to invoke multiple tasks in a single command line, e.g. ``fab task1 task2`` - Ported - Works great! * - ``python -m fabric`` as stand-in for ``fab`` - Ported - Ported in 2.2. * - ``-a``/``--no_agent`` for disabling automatic SSH agent key selection - Removed - To disable use of an agent permanently, set config value ``connect_kwargs.allow_agent`` to ``False``; to disable temporarily, unset the ``SSH_AUTH_SOCK`` env var. * - ``-A``/``--forward-agent`` for enabling agent forwarding to the remote end - Removed - The config and kwarg versions of this are ported, but there is currently no CLI flag. Usual "you can set the config value at runtime with a shell env variable" clause is in effect, so this *may* not get ported, depending. * - ``--abort-on-prompts`` to turn interactive prompts into exceptions (helps avoid 'hanging' sessions) - Removed - See the notes about interactive prompts going away in :ref:`upgrading-general`. Without mid-session prompts, there's no need for this option. * - ``-c``/``--config`` for specifying an alternate config file path - Ported - ``--config`` lives on, but the short flag is now ``-f`` (``-c`` now determines which collection module name is sought by the task loader.) * - ``--colorize-errors`` (and ``env.colorize_errors``) to enable ANSI coloring of error output - `Pending `__ - Very little color work has been done yet and this is one of the potentially missing pieces. We're unsure how often this was used in v1 so it's possible it won't show up again, but generally, we like using color as an additional output vector, so... * - ``-d``/``--display`` for showing info on a given command - Ported - This is now the more standard ``-h``/``--help``, and can be given in either "direction": ``fab -h mytask`` or ``fab mytask -h``. * - ``-D``/``--disable-known-hosts`` to turn off Paramiko's automatic loading of user-level ``known_hosts`` files - `Pending `__ - Not ported yet, probably will be. * - ``-e``/``--eagerly-disconnect`` (and ``env.eagerly_disconnect``) which tells the execution system to disconnect from hosts as soon as a task is done running - Ported/`Pending `__ - There's no explicit connection cache anymore, so eager disconnection should be less necessary. However, investigation and potential feature toggles are still pending. * - ``-f``/``--fabfile`` to select alternate fabfile location - Ported - This is now split up into ``-c``/``--collection`` and ``-r``/``--search-root``; see :ref:`loading-collections`. * - ``-g``/``--gateway`` (and ``env.gateway``) for selecting a global SSH gateway host string - `Pending `__ - One can set the global ``gateway`` config option via an environment variable, which at a glance would remove the need for a dedicated CLI option. However, this approach only allows setting string values, which in turn only get used for ``ProxyCommand`` style gatewaying, so it *doesn't* replace v1's ``--gateway`` (which took a host string and turned it into a ``ProxyJump`` style gateway). Thus, if enough users notice the lack, we'll consider a feature-add that largely mimics the v1 behavior: string becomes first argument to `fabric.connection.Connection` and that resulting object is then set as ``gateway``. * - ``--gss-auth``/``--gss-deleg``/``--gss-kex`` - Removed - These didn't seem used enough to be worth porting over, especially since they fall under the usual umbrella of "Paramiko-level connect passthrough" covered by the ``connect_kwargs`` config option. (Which, if necessary, can be set at runtime via shell environment variables, like any other config value.) * - ``--hide``/``--show`` for tweaking output display globally - Removed - This is configurable via the config system and env vars. * - ``-H``/``--hosts`` - Ported - Works basically the same as before - if given, is shorthand for executing any given tasks once per host. * - ``-i`` for SSH key filename selection - Ported - Works same as v1, including ability to give multiple times to build a list of keys to try. * - ``-I``/``--initial-password-prompt`` for requesting an initial pre-execution password prompt - Ported - It's now :option:`--prompt-for-login-password`, :ref:`--prompt-for-sudo-password ` or :option:`--prompt-for-passphrase`, depending on whether you were using the former to fill in passwords or key passphrases (or both.) * - ``--initial-sudo-password-prompt`` for requesting an initial pre-execution sudo password prompt - Ported - This is now :option:`--prompt-for-sudo-password`. Still a bit of a mouthful but still 4 characters shorter! * - ``-k``/``--no-keys`` which prevents Paramiko's automatic loading of key files such as ``~/.ssh/id_rsa`` - Removed - Use environment variables to set the ``connect_kwargs.look_for_keys`` config value to ``False``. * - ``--keepalive`` for setting network keepalive - `Pending `__ - Not ported yet. * - ``-l``/``--list`` for listing tasks, plus ``-F``/``--list-format`` for tweaking list display format - Ported - Now with bonus JSON list-format! Which incidentally replaces ``-F short``/``--shortlist``. * - ``--linewise`` for buffering output line by line instead of roughly byte by byte - Removed - This doesn't really fit with the way modern command execution code views the world, so it's gone. * - ``-n``/``--connection-attempts`` controlling multiple connect retries - `Pending `__ - Not ported yet. * - ``--no-pty`` to disable automatic PTY allocation in ``run``, etc - Ported - Is now ``-p``/``--pty`` as the default behavior was switched around. * - ``--password``/``--sudo-password`` for specifying login/sudo password values - Removed - This is typically not very secure to begin with, and there are now many other avenues for setting the related configuration values, so they're gone at least for now. * - ``-P``/``--parallel`` for activating global parallelism - `Pending `__ - See the notes around ``@parallel`` in :ref:`upgrading-tasks`. * - ``--port`` to set default SSH port - Removed - Our gut says this is best left up to the configuration system's env var layer, or use of the ``port`` kwarg on `fabric.connection.Connection`; however it may find its way back. * - ``r``/``--reject-unknown-hosts`` to modify Paramiko known host behavior - `Pending `__ - Not ported yet. * - ``-R``/``--roles`` for global list-of-hosts target selection - `Pending `__ - As noted under :ref:`upgrading-api`, role lists are only partially applicable to the new API and we're still feeling out whether/how they would work at a global or CLI level. * - ``--set key=value`` for setting ``fabric.state.env`` vars at runtime - Removed - This is largely obviated by the new support for shell environment variables (just do ``INVOKE_KEY=value fab mytask`` or similar), though it's remotely possible a CLI flag method of setting config values will reappear later. * - ``-s``/``--shell`` to override default shell path - Removed - Use the configuration system for this. * - ``--shortlist`` for short/computer-friendly list output - Ported - See ``--list``/``--list-format`` - there's now a JSON format instead. No point reinventing the wheel. * - ``--skip-bad-hosts`` (and ``env.skip_bad_hosts``) to bypass problematic hosts - `Pending `__ - Not ported yet. * - ``--skip-unknown-tasks`` and ``env.skip_unknown_tasks`` for silently skipping past bogus task names on CLI invocation - Removed - This felt mostly like bloat to us and could require nontrivial parser changes to reimplement, so it's out for now. * - ``--ssh-config-path`` and ``env.ssh_config_path`` for selecting an SSH config file - Ported - This is now ``-S``/``--ssh-config``. * - ``--system-known-hosts`` to trigger loading systemwide ``known_hosts`` files - `Pending `__/Removed - This isn't super likely to come back as its own CLI flag but it may well return as a configuration value. * - ``-t``/``--timeout`` controlling connection timeout - Ported - It's now ``-t``/``--connect-timeout`` as ``--timeout`` was technically ambiguous re: connect vs command timeout. * - ``-T``/``--command-timeout`` - Ported - Implemented in Invoke and preserved in ``fab`` under the same name. * - ``-u``/``--user`` to set global default username - Removed - Most of the time, configuration (env vars for true runtime, or eg user/project level config files as appropriate) should be used for this, but it may return. * - ``-w``/``--warn-only`` to toggle warn-vs-abort behavior - Ported - Ported as-is, no changes. * - ``-x``/``--exclude-hosts`` (and ``env.exclude_hosts``) for excluding otherwise selected targets - `Pending `__ - Not ported yet, is pending an in depth rework of global (vs hand-instantiated) connection/group selection. * - ``-z``/``--pool-size`` for setting parallel-mode job queue pool size - Removed - There's no job queue anymore, or at least at present. Whatever replaces it (besides the already-implemented threading model) is likely to look pretty different. .. _upgrading-commands: Shell command execution (``local``/``run``/``sudo``) ---------------------------------------------------- General ~~~~~~~ Behaviors shared across either ``run``/``sudo``, or all of ``run``/``sudo``/``local``. Subsequent sections go into per-function differences. .. list-table:: :widths: 40 10 50 * - ``local`` and ``run``/``sudo`` have wildly differing APIs and implementations - Removed - All command execution is now unified; all three functions (now methods on `fabric.connection.Connection`, though ``local`` is also available as `invoke.run` for standalone use) have the same underlying protocol and logic (the `~invoke.runners.Runner` class hierarchy), with only low-level details like process creation and pipe consumption differing. For example, in v1 ``local`` required you to choose between displaying and capturing subprocess output; modern ``local`` is like ``run`` and does both at the same time. * - Prompt auto-response, via ``env.prompts`` and/or ``sudo``'s internals - Ported - The ``env.prompts`` functionality has been significantly fleshed out, into a framework of :ref:`Watchers ` which operate on any (local or remote!) running command's input and output streams. In addition, ``sudo`` has been rewritten to use that framework; while still useful enough to offer an implementation in core, it no longer does anything users cannot do themselves using public APIs. * - ``fabric.context_managers.cd``/``lcd`` (and ``prefix``) allow scoped mutation of executed comments - Ported/`Pending `__ - These are now methods on `~invoke.context.Context` (`Context.cd `, `Context.prefix `) but need work in its subclass `fabric.connection.Connection` (quite possibly including recreating ``lcd``) so that local vs remote state are separated. * - ``fabric.context_managers.shell_env`` and its specific expression ``path`` (plus ``env.shell_env``, ``env.path`` and ``env.path_behavior``), for modifying remote environment variables (locally, one would just modify `os.environ`.) - Ported - The context managers were the only way to set environment variables at any scope; in modern Fabric, subprocess shell environment is controllable per-call (directly in `fabric.connection.Connection.run` and siblings via an ``env`` kwarg) *and* across multiple calls (by manipulating the configuration system, statically or at runtime.) * - Controlling subprocess output & other activity display text by manipulating ``fabric.state.output`` (directly or via ``fabric.context_managers.hide``, ``show`` or ``quiet`` as well as the ``quiet`` kwarg to ``run``/``sudo``; plus ``utils.puts``/``fastprint``) - Ported/`Pending `__ - The core concept of "output levels" is gone, likely to be replaced in the near term by a logging module (stdlib or other) which output levels poorly reimplemented. Command execution methods like `~invoke.runners.Runner.run` retain a ``hide`` kwarg controlling which subprocess streams are copied to your terminal, and an ``echo`` kwarg controlling whether commands are printed before execution. All of these also honor the configuration system. * - ``timeout`` kwarg and the ``CommandTimeout`` exception raised when said command-runtime timeout was violated - Ported - Primarily lives at the Invoke layer now, but applies to all command execution, local or remote; see the ``timeout`` argument to `~invoke.runners.Runner.run` and its related configuration value and CLI flag. * - ``pty`` kwarg and ``env.always_use_pty``, controlling whether commands run in a pseudo-terminal or are invoked directly - Ported - This has been thoroughly ported (and its behavior often improved) including preservation of the ``pty`` kwarg and updating the config value to be simply ``run.pty``. However, a major change is that pty allocation is now ``False`` by default instead of ``True``. Fabric 0.x and 1.x already changed this value around; during Fabric 1's long lifetime it became clear that neither default works for all or even most users, so we opted to return the default to ``False`` as it's cleaner and less wasteful. * - ``combine_stderr`` (kwarg and ``env.combine_stderr``) controlling whether Paramiko weaves remote stdout and stderr into the stdout stream - Removed - This wasn't terrifically useful, and often caused conceptual problems in tandem with ``pty`` (as pseudo-terminals by their nature always combine the two streams.) We recommend users who really need both streams to be merged, either use shell redirection in their command, or set ``pty=True``. * - ``warn_only`` kwarg for preventing automatic abort on non-zero return codes - Ported - This is now just ``warn``, both kwarg and config value. It continues to default to ``False``. * - ``stdout`` and ``stderr`` kwargs for reassigning default stdout/err mirroring targets, which otherwise default to the appropriate `sys` members - Ported - These are now ``out_stream`` and ``err_stream`` but otherwise remain similar in nature. They are also accompanied by the new, rather obvious in hindsight ``in_stream``. * - ``capture_buffer_size`` arg & use of a ring buffer for storing captured stdout/stderr to limit total size - `Pending `__ - Existing `~invoke.runners.Runner` implementation uses regular lists for capture buffers, but we fully expect to upgrade this to a ring buffer or similar at some point. * - Return values are string-like objects with extra attributes like ``succeeded`` and ``return_code`` sprinkled on top - Ported - Return values are no longer string-a-likes with a semi-private API, but are full fledged regular objects of type `~invoke.runners.Result`. They expose all of the same info as the old "attribute strings", and only really differ in that they don't pretend to be strings themselves. They do, however, still behave as booleans - just ones reflecting the exit code's relation to zero instead of whether there was any stdout. * - ``open_shell`` for obtaining interactive-friendly remote shell sessions (something that ``run`` historically was bad at ) - Ported - Not only is the new version of ``run`` vastly improved and able to deal with interactive sessions at least as well as the old ``open_shell`` (provided you supply ``pty=True``), but for corner cases there's also a direct port: `~fabric.connection.Connection.shell`. ``run`` ~~~~~~~ .. list-table:: :widths: 40 10 50 * - ``shell`` / ``env.use_shell`` designating whether or not to wrap commands within an explicit call to e.g. ``/bin/sh -c "real command"``; plus their attendant options like ``shell_escape`` - Removed - Non-``sudo`` remote execution never truly required an explicit shell wrapper: the remote SSH daemon hands your command string off to the connecting user's login shell in almost all cases. Since wrapping is otherwise extremely error-prone and requires frustrating escaping rules, we dropped it for this use case. See the matching line items for ``local`` and ``sudo`` as their situations differ. (For now, because they all share the same underpinnings, `fabric.connection.Connection.run` does accept a ``shell`` kwarg - it just doesn't do anything with it.) ``sudo`` ~~~~~~~~ Unless otherwise noted, all common ``run``+``sudo`` args/functionality (e.g. ``pty``, ``warn_only`` etc) are covered above in the section on ``run``; the below are ``sudo`` specific. .. list-table:: :widths: 40 10 50 * - ``shell`` / ``env.use_shell`` designating whether or not to wrap commands within an explicit call to e.g. ``/bin/sh -c "real command"`` - `Pending `__/Removed - See the note above under ``run`` for details on shell wrapping as a general strategy; unfortunately for ``sudo``, some sort of manual wrapping is still necessary for nontrivial commands (i.e. anything using actual shell syntax as opposed to a single program's argv) due to how the command string is handed off to the ``sudo`` program. We hope to upgrade ``sudo`` soon so it can perform a common-best-case, no-escaping-required shell wrapping on your behalf; see the 'Pending' link. * - ``user`` argument (and ``env.sudo_user``) allowing invocation via ``sudo -u `` (instead of defaulting to root) - Ported - This is still here, and still called ``user``. * - ``group`` argument controlling the effective group of the sudo'd command - `Pending `__ - This has not been ported yet. ``local`` ~~~~~~~~~ See the 'general' notes at top of this section for most details about the new ``local``. A few specific extras are below. .. list-table:: :widths: 40 10 50 * - ``shell`` kwarg designating which shell to ask `subprocess.Popen` to use - Ported - Basically the same as in v1, though there are now situations where `os.execve` (or similar) is used instead of `subprocess.Popen`. Behavior is much the same: no shell wrapping (as in legacy ``run``), just informing the operating system what actual program to run. ``open_shell`` ~~~~~~~~~~~~~~ As noted in the main list, this is now `~fabric.connection.Connection.shell`, and behaves similarly to ``open_shell`` (exit codes, if any, are ignored; a PTY is assumed; etc). It has some improvements too, such as a return value (which is slightly lacking compared to that from `~fabric.connection.Connection.run` but still a big improvement over ``None``). .. list-table:: :widths: 40 10 50 * - ``command`` optional kwarg allowing 'prefilling' the input stream with a specific command string plus newline - Removed - If you needed this, you should instead try the modern version of `~fabric.connection.Connection.run`, which is equally capable of interaction as `~fabric.connection.Connection.shell` but takes a command to execute. There's a small chance we'll add this back later if anybody misses it (there's a few corner cases that could possibly want it). .. _upgrading-utility: Utilities --------- .. list-table:: :widths: 40 10 50 * - Error handling via ``abort`` and ``warn`` - Ported - The old functionality leaned too far in the "everything is a DSL" direction & didn't offer enough value to offset how it gets in the way of experienced Pythonistas. These functions have been removed in favor of "just raise an exception" (with one useful option being Invoke's `~invoke.exceptions.Exit`) as exception handling feels more Pythonic than thin wrappers around ``sys.exit`` or having to ``except SystemExit:`` and hope it was a `SystemExit` your own code raised! * - ANSI color helpers in ``fabric.colors`` allowed users to easily print ANSI colored text without a standalone library - Removed - There seemed no point to poorly replicating one of the many fine terminal-massaging libraries out there (such as those listed in the description of `#101 `_) in the rewrite, so we didn't. That said, it seems highly plausible we'll end up vendoring such a library in the future to offer internal color support, at which point "baked-in" color helpers would again be within easy reach. * - ``with char_buffered`` context manager for forcing a local stream to be character buffered - Ported - This is now `~invoke.terminals.character_buffered`. * - ``docs.unwrap_tasks`` for extracting docstrings from wrapped task functions - Ported - v1 required using a Fabric-specific 'unwrap_tasks' helper function somewhere in your Sphinx build pipeline; now you can instead just enable the new `invocations.autodoc `_ Sphinx mini-plugin in your extensions list; see link for details. * - ``network.normalize``, ``denormalize`` and ``parse_host_string``, ostensibly internals but sometimes exposed to users for dealing with host strings - Removed - As with other host-string-related tools, these are gone and serve no purpose. `fabric.connection.Connection` is now the primary API focus and has individual attributes for all "host string" components. * - ``utils.indent`` for indenting/wrapping text (uncommonly used) - Pending - Not ported yet; ideally we'll just vendor a third party lib in Invoke. * - ``reboot`` for rebooting and reconnecting to a remote system - Removed - No equivalent has been written for modern Fabric; now that the connection/client objects are made explicit, one can simply instantiate a new object with the same parameters (potentially with sufficient timeout parameters to get past the reboot, if one doesn't want to manually call something like `time.sleep`.) There is a small chance it will return if there appears to be enough need; if so, it's likely to be a more generic reconnection related `fabric.connection.Connection` method, where the user is responsible for issuing the restart shell command via ``sudo`` themselves. * - ``require`` for ensuring certain key(s) in ``env`` have values set, optionally by noting they can be ``provided_by=`` a list of setup tasks - Removed - This has not been ported, in part because the maintainers never used it themselves, and is unlikely to be directly reimplemented. However, its core use case of "require certain data to be available to run a given task" may return within the upcoming dependency framework. * - ``prompt`` for prompting the user & storing the entered data (optionally with validation) directly into ``env`` - Removed - Like ``require``, this seemed like a less-used feature (especially compared to its sibling ``confirm``) and was not ported. If it returns it's likely to be via ``invocations``, which is where ``confirm`` ended up. .. _upgrading-networking: Networking ---------- .. list-table:: :widths: 40 10 50 * - ``env.gateway`` for setting an SSH jump gateway - Ported - This is now the ``gateway`` kwarg to `fabric.connection.Connection`, and -- for the newly supported ``ProxyJump`` style gateways, which can be nested indefinitely! -- should be another `fabric.connection.Connection` object instead of a host string. (You may specify a runtime, non-SSH-config-driven ``ProxyCommand``-style string as the ``gateway`` kwarg instead, which will act just like a regular ``ProxyCommand``.) * - ``ssh_config``-driven ``ProxyCommand`` support - Ported - This continues to work as it did in v1. * - ``with remote_tunnel(...):`` port forwarding - Ported - This is now `fabric.connection.Connection.forward_local`, since it's used to *forward* a *local* port to the remote end. (Newly added is the logical inverse, `fabric.connection.Connection.forward_remote`.) * - ``NetworkError`` raised on some network related errors - Removed - In v1 this was simply a (partially implemented) stepping-back from the original "just sys.exit on any error!" behavior. Modern Fabric is significantly more exception-friendly; situations that would raise ``NetworkError`` in v1 now simply become the real underlying exceptions, typically from Paramiko or the stdlib. * - ``env.keepalive`` for setting network keepalive value - `Pending `__ - Not ported yet. * - ``env.connection_attempts`` for setting connection retries - `Pending `__ - Not ported yet. * - ``env.timeout`` for controlling connection (and sometimes command execution) timeout - Ported - Connection timeout is now controllable both via the configuration system (as ``timeouts.connect``) and a direct kwarg on `fabric.connection.Connection`. Command execution timeout is its own setting now, ``timeouts.command`` and a ``timeout`` kwarg to ``run`` and friends. Authentication -------------- .. note:: Some ``env`` keys from v1 were simply passthroughs to Paramiko's `SSHClient.connect ` method. Modern Fabric gives you explicit control over the arguments it passes to that method, via the ``connect_kwargs`` :ref:`configuration ` subtree, and the below table will frequently refer you to that approach. .. list-table:: :widths: 40 10 50 * - ``env.key_filename`` - Ported - Use ``connect_kwargs``. * - ``env.password`` - Ported - Use ``connect_kwargs``. Also note that this used to perform double duty as connection *and* sudo password; the latter is now found in the ``sudo.password`` setting. * - ``env.gss_(auth|deleg|kex)`` - Ported - Use ``connect_kwargs``. * - ``env.key``, a string or file object holding private key data, whose specific type is auto-determined and instantiated for use as the ``pkey`` connect kwarg - Removed - This has been dropped as unnecessary (& bug-prone) obfuscation of Paramiko-level APIs; users should already know which type of key they're dealing with and instantiate a ``PKey`` subclass themselves, placing the result in ``connect_kwargs.pkey``. * - ``env.no_agent``, which is a renaming/inversion of Paramiko's ``allow_agent`` connect kwarg - Ported - Users who were setting this to ``True`` should now simply set ``connect_kwargs.allow_agent`` to ``False`` instead. * - ``env.no_keys``, similar to ``no_agent``, just an inversion of the ``look_for_keys`` connect kwarg - Ported - Use ``connect_kwargs.look_for_keys`` instead (setting it to ``False`` to disable Paramiko's default key-finding behavior.) * - ``env.passwords`` (and ``env.sudo_passwords``) stores connection/sudo passwords in a dict keyed by host strings - Ported/`Pending `__ - Each `fabric.connection.Connection` object may be configured with its own ``connect_kwargs`` given at instantiation time, allowing for per-host password configuration already. However, we expect users may want a simpler way to set configuration values that are turned into implicit `fabric.connection.Connection` objects automatically; such a feature is still pending. * - Configuring ``IdentityFile`` in one's ``ssh_config`` - Ported - Still honored, along with a bunch of newly honored ``ssh_config`` settings; see :ref:`ssh-config`. .. _upgrading-transfers: File transfer ------------- The below feature breakdown applies to the ``put`` and/or ``get`` "operation" functions from v1. .. list-table:: :widths: 40 10 50 * - Transferring individual files owned by the local and remote user - Ported - Basic file transfer in either direction works and is offered as `fabric.connection.Connection.get`/`fabric.connection.Connection.put` (though the code is split out into a separate-responsibility class, `fabric.transfer.Transfer`.) The signature of these methods has been cleaned up compared to v1, though their positional-argument essence (``get(remote, local)`` and ``put(local, remote)`` remains the same. * - Omit the 'destination' argument for implicit 'relative to local context' behavior (e.g. ``put("local.txt")`` implicitly uploading to remote ``$HOME/local.txt``.) - Ported - You should probably still be explicit, because this is Python. * - Use either file paths *or* file-like objects on either side of the transfer operation (e.g. uploading a ``StringIO`` instead of an on-disk file) - Ported - This was a useful enough and simple enough trick to keep around. * - Preservation of source file mode at destination (e.g. ensuring an executable bit that would otherwise be dropped by the destination's umask, is re-added.) - Ported - Not only was this ported, but it is now the default behavior. It may be disabled via kwarg if desired. * - Bundled ``sudo`` operations as part of file transfer - Removed - This was one of the absolute buggiest parts of v1 and never truly did anything users could not do themselves with a followup call to ``sudo``, so we opted not to port it. Should enough users pine for its loss, we *may* reconsider, but if we do it will be with a serious eye towards simplification and/or an approach not involving intermediate files. * - Recursive multi-file transfer (e.g. ``put(a_directory)`` uploads entire directory and all its contents) - Removed - This was *another* one of the buggiest parts of v1, and over time it became clear that its maintenance burden far outweighed the fact that it was poorly reinventing ``rsync`` and/or the use of archival file tools like ye olde ``tar``+``gzip``. For one potential workaround, see the ``rsync`` function in `patchwork `_. * - Remote file path tilde expansion - Removed - This behavior is ultimately unnecessary (one can simply leave the tilde off for the same result) and had a few pernicious bugs of its own, so it's gone. * - Naming downloaded files after some aspect of the remote destination, to avoid overwriting during multi-server actions - Ported - Added back (to `fabric.transfer.Transfer.get`) in Fabric 2.6. .. _upgrading-configuration: Configuration ------------- In general, configuration has been massively improved over the old ``fabricrc`` files; most config logic comes from :ref:`Invoke's configuration system `, which offers a full-fledged configuration hierarchy (in-code config, multiple config file locations, environment variables, CLI flags, and more) and multiple file formats. Nearly all configuration avenues in Fabric 1 become, in modern Fabric, manipulation of whatever part of the config hierarchy is most appropriate for your needs. Modern versions of Fabric only make minor modifications to (or parameterizations of) Invoke's setup; see :ref:`our locally-specific config doc page ` for details. .. note:: Make sure to look elsewhere in this document for details on any given v1 ``env`` setting, as many have moved outside the configuration system into object or method keyword arguments. .. list-table:: :widths: 40 10 50 * - Modifying ``fabric.(api.)env`` directly - Ported - To effect truly global-scale config changes, use config files, task-collection-level config data, or the invoking shell's environment variables. * - Making locally scoped ``fabric.env`` changes via ``with settings(...):`` or its decorator equivalent, ``@with_settings`` - Ported/Pending - Most of the use cases surrounding ``settings`` are now served by the fact that `fabric.connection.Connection` objects keep per-host/connection state - the pattern of switching the implicit global context around was a design antipattern which is now gone. The remaining such use cases have been turned into context-manager methods of `fabric.connection.Connection` (or its parent class), or have such methods pending. * - SSH config file loading (off by default, limited to ``~/.ssh/config`` only unless configured to a different, single path) - Ported - Much improved: SSH config file loading is **on** by default (which :ref:`can be changed `), multiple sources are loaded and merged just like OpenSSH, and more besides; see :ref:`ssh-config`. In addition, we've added support for some ``ssh_config`` directives which were ignored by v1, such as ``ConnectTimeout`` and ``ProxyCommand``, and going forwards we intend to support as much of ``ssh_config`` as is reasonably possible. .. _upgrading-contrib: ``contrib`` ----------- The old ``contrib`` module represented "best practice" functions that did not, themselves, require core support from the rest of Fabric but were built using the same primitives available to users. In modern Fabric, that responsibility has been removed from the core library into other standalone libraries which have their own identity & release process, typically either `invocations `_ (local-oriented code that does not use SSH) or `patchwork `_ (primarily remote-oriented code, though anything not explicitly dealing with both ends of the connection will work just as well locally.) Those libraries are still a work in progress, not least because we still need to identify the best way to bridge the gap between them (as many operations are not intrinsically local-or-remote but can work on either end.) Since they are by definition built on the core APIs available to all users, they currently get less development focus; users can always implement their own versions without sacrificing much (something less true for the core libraries.) We expect to put more work into curating these collections once the core APIs have settled down. Details about what happened to each individual chunk of ``fabric.contrib`` are in the below table: .. list-table:: :widths: 40 10 50 * - ``console.confirm`` for easy bool-returning confirmation prompts - Ported - Moved to ``invocations.console.confirm``, with minor signature tweaks. * - ``django.*``, supporting integration with a local Django project re: importing and using Django models and other code - Removed - We aren't even sure if this is useful a decade after it was written, given how much Django has surely changed since then. If you're reading and are sad that this is gone, let us know! * - ``files.*`` (e.g. ``exists``, ``append``, ``contains`` etc) for interrogating and modifying remote files - Ported/Pending - Many of the more useful functions in this file have been ported to ``patchwork.files`` but are still in an essentially alpha state. Others, such as ``is_link``, ``comment``/``uncomment``, etc have not been ported yet. If they are, the are likely to end up in the same place. * - ``project.rsync_project`` for rsyncing the entire host project remotely - Ported - Now ``patchwork.transfers.rsync``, with some modifications. * - ``project.rsync_project`` for uploading host project via archive file and scp - Removed - This did not seem worth porting; the overall pattern of "copy my local bits remotely" is already arguably an antipattern (vs repeatable deploys of artifacts, or at least remote checkout of a VCS tag) and if one is going down that road anyways, rsync is a much smarter choice. .. _upgrading-env: ``fabric.env`` reference ------------------------ Many/most of the members in v1's ``fabric.env`` are covered in the above per-topic sections; any that are *not* covered elsewhere, live here. All are explicitly noted as ``env.`` for ease of searching in your browser or viewer. A small handful of env vars were never publicly documented & were thus implicitly private; those are not represented here. .. list-table:: :widths: 40 10 50 * - ``env.abort_exception`` for setting which exception is used to abort - Removed - Aborting as a concept is gone, just raise whatever exception seems most reasonable to surface to an end user, or use `~invoke.exceptions.Exit`. See also :ref:`upgrading-utility`. * - ``env.all_hosts`` and ``env.tasks`` listing execution targets - Ported/`Pending `__ - Fabric's `~invoke.executor.Executor` subclass stores references to all CLI parsing results (including the value of :option:`--hosts`, the tasks requested and their args, etc) and the intent is for users to have access to that information. However, the details for that API (e.g. exposing the executor via a task's `~invoke.context.Context`/`fabric.connection.Connection`) are still in flux. * - ``env.command`` noting currently executing task name (in hindsight, quite the misnomer...) - Ported/`Pending `__ - See the notes for ``env.all_hosts`` above - same applies here re: user visibility into CLI parsing results. * - ``env.command_prefixes`` for visibility into (arguably also mutation of) the shell command prefixes to be applied to ``run``/``sudo`` - Ported - This is now `~invoke.context.Context.command_prefixes`. * - ``env.cwd`` noting current intended working directory - Ported - This is now `~invoke.context.Context.command_cwds` (a list, not a single string, to more properly model the intended contextmanager-driven use case.) Note that remote-vs-local context for this data isn't yet set up; see the notes about ``with cd`` under :ref:`upgrading-commands`. * - ``env.dedupe_hosts`` controlling whether duplicate hosts in merged host lists get deduplicated or not - `Pending `__ - Not ported yet, will probably get tackled as part of roles/host lists overhaul. * - ``env.echo_stdin`` (undocumented) for turning off the default echoing of standard input - Ported - Is now a config option under the ``run`` tree, with much the same behavior. * - ``env.local_user`` for read-only access to the discovered local username - Removed - We're not entirely sure why v1 felt this was worth caching in the config; if you need this info, just import and call `fabric.util.get_local_user`. * - ``env.output_prefix`` determining whether or not line-by-line host-string prefixes are displayed - `Pending `__ - Differentiating parallel stdout/err is still a work in progress; we may end up reusing line-by-line logging and prefixing (ideally via actual logging) or we may try for something cleaner such as streaming to per-connection log files. * - ``env.prompts`` controlling prompt auto-response - Ported - Prompt auto-response is now publicly implemented as the `~invoke.watchers.StreamWatcher` and `~invoke.watchers.Responder` class hierarchy, instances of which can be handed to ``run`` via kwarg or stored globally in the config as ``run.watchers``. * - ``env.real_fabfile`` storing read-only fabfile path which was loaded by the CLI machinery - Ported - The loaded task `~invoke.collection.Collection` is stored on both the top level `~invoke.program.Program` object as well as the `~invoke.executor.Executor` which calls tasks; and `~invoke.collection.Collection` has a ``loaded_from`` attribute with this information. * - ``env.remote_interrupt`` controlling how interrupts (i.e. a local `KeyboardInterrupt` are caught, forwarded or other - Ported/Removed - Invoke's interrupt capture behavior is currently "always just send the interrupt character to the subprocess and continue", allowing subprocesses to handle ``^C`` however they need to, which is an improvement over Fabric 1 and roughly equivalent to setting ``env.remote_interrupt = True``. Allowing users to change this behavior via config is not yet implemented, and may not be, depending on whether anybody needs it - it was added as an option in v1 for backwards compat reasons. It is also technically possible to change interrupt behavior by subclassing and overriding `invoke.runners.Runner.send_interrupt`. * - ``env.roles``, ``env.roledefs`` and ``env.effective_roles`` controlling/exposing what roles are available or currently in play - `Pending `__ - As noted in :ref:`upgrading-api`, roles as a concept were ported to `fabric.group.Group`, but there's no central clearinghouse in which to store them. We *may* delegate this to userland forever, but seems likely a common-best-practice option (such as creating `Groups ` from some configuration subtree and storing them as a `~invoke.context.Context` attribute) will appear in early 2.x. * - ``env.ok_ret_codes`` for overriding the default "0 good, non-0 bad" error detection for subprocess commands - `Pending `__ - Not ported yet, but should involve some presumably minor updates to `invoke.runners.Runner.generate_result` and `~invoke.runners.Result`. * - ``env.sudo_prefix`` determining the sudo binary name + its flags used when creating ``sudo`` command strings - `Pending `__ - Sudo command construction does not currently look at the config for anything but the actual sudo prompt. * - ``env.sudo_prompt`` for setting the prompt string handed to ``sudo`` (and then expected in return for auto-replying with a configured password) - Ported - Is now ``sudo.prompt`` in the configuration system. * - ``env.use_exceptions_for`` to note which actions raise exceptions - Removed - As with most other functionality surrounding Fabric 1's "jump straight to `sys.exit`" design antipattern, this is gone - modern Fabric will not be hiding any exceptions from user-level code. * - ``env.use_ssh_config`` to enable off-by-default SSH config loading - Ported - SSH config loading is now on by default, but an option remains to disable it. See :ref:`upgrading-configuration` for more. * - ``env.version`` exposing current Fabric version number - Removed - Just ``import fabric`` and reference ``fabric.__version__`` (string) or ``fabric.__version_info__`` (tuple). Example upgrade process ======================= This section goes over upgrading a small but nontrivial Fabric 1 fabfile to work with modern Fabric. It's not meant to be exhaustive, merely illustrative; for a full list of how to upgrade individual features or concepts, see :ref:`upgrade-specifics`. Sample original fabfile ----------------------- Here's a (slightly modified to concur with 'modern' Fabric 1 best practices) copy of Fabric 1's final tutorial snippet, which we will use as our test case for upgrading:: from fabric.api import abort, env, local, run, settings, task from fabric.contrib.console import confirm env.hosts = ["my-server"] @task def test(): with settings(warn_only=True): result = local("./manage.py test my_app", capture=True) if result.failed and not confirm("Tests failed. Continue anyway?"): abort("Aborting at user request.") @task def commit(): local("git add -p && git commit") @task def push(): local("git push") @task def prepare_deploy(): test() commit() push() @task def deploy(): code_dir = "/srv/django/myproject" with settings(warn_only=True): if run("test -d {}".format(code_dir)).failed: cmd = "git clone user@vcshost:/path/to/repo/.git {}" run(cmd.format(code_dir)) with cd(code_dir): run("git pull") run("touch app.wsgi") We'll port this directly, meaning the result will still be ``fabfile.py``, though we'd like to note that writing your code in a more library-oriented fashion - even just as functions not wrapped in ``@task`` - can make testing and reusing code easier. Imports ------- In modern Fabric, we don't need to import nearly as many functions, due to the emphasis on object methods instead of global functions. We only need the following: - `~invoke.exceptions.Exit`, a friendlier way of requesting a `sys.exit`; - `@task `, as before, but coming from Invoke as it's not SSH-specific; - ``confirm``, which now comes from the Invocations library (also not SSH-specific; though Invocations is one of the descendants of ``fabric.contrib``, which no longer exists); :: from fabric import task from invoke import Exit from invocations.console import confirm Host list --------- The idea of a predefined *global* host list is gone; there is currently no direct replacement. In general, users can set up their own execution context, creating explicit `fabric.connection.Connection` and/or `fabric.group.Group` objects as needed; core Fabric is in the process of building convenience helpers on top of this, but "create your own Connections" will always be there as a backstop. Speaking of convenience helpers: most of the functionality of ``fab --hosts`` and ``@hosts`` has been ported over -- the former directly (see :option:`--hosts`), the latter as a `@task ` keyword argument. Thus, for now our example will be turning the global ``env.hosts`` into a lightweight module-level variable declaration, intended for use in the subsequent calls to ``@task``:: my_hosts = ["my-server"] .. note:: This is an area under active development, so feedback is welcomed. .. TODO: - pre-task example - true baked-in default example (requires some sort of config hook) Test task --------- The first task in the fabfile uses a good spread of the API. We'll outline the changes here (though again, all details are in :ref:`upgrade-specifics`): - Declaring a function as a task is nearly the same as before: use a ``@task`` decorator (which, in modern Fabric, can take more optional keyword arguments than its predecessor, including some which replace some of v1's decorators). - ``@task``-wrapped functions must now take an explicit initial context argument, whose value will be a `fabric.connection.Connection` object at runtime. - The use of ``with settings(warn_only=True)`` can be replaced by a simple kwarg to the ``local`` call. - That ``local`` call is now a method call on the `fabric.connection.Connection`, `fabric.connection.Connection.local`. - ``capture`` is no longer a useful argument; we can now capture and display at the same time, locally or remotely. If you don't actually *want* a local subprocess to mirror its stdout/err while it runs, you can simply say ``hide=True`` (or ``hide="stdout"`` or etc.) - Result objects are pretty similar between versions; modern Fabric's results no longer pretend to "be" strings, but instead act more like booleans, acting truthy if the command exited cleanly, and falsey otherwise. In terms of attributes exhibited, most of the same info is available, and more besides. - ``abort`` is gone; you should use whatever exceptions you feel are appropriate, or `~invoke.exceptions.Exit` for a `sys.exit` equivalent. (Or just call `sys.exit` if you want a no-questions-asked immediate exit that even our CLI machinery won't touch.) The result:: @task def test(c): result = c.local("./manage.py test my_app", warn=True) if not result and not confirm("Tests failed. Continue anyway?"): raise Exit("Aborting at user request.") Other simple tasks ------------------ The next two tasks are simple one-liners, and you've already seen what replaced the global ``local`` function:: @task def commit(c): c.local("git add -p && git commit") @task def push(c): c.local("git push") Calling tasks from other tasks ------------------------------ This is another area that is in flux at the Invoke level, but for now, we can simply call the other tasks as functions, just as was done in v1. The main difference is that we want to pass along our context object to preserve the configuration context (such as loaded config files or CLI flags):: @task def prepare_deploy(c): test(c) commit(c) push(c) Actual remote steps ------------------- Note that up to this point, nothing truly Fabric-related has been in play - `fabric.connection.Connection.local` is just a rebinding of `Context.run `, Invoke's local subprocess execution method. Now we get to the actual deploy step, which invokes `fabric.connection.Connection.run` instead, executing remotely (on whichever host the `fabric.connection.Connection` has been bound to). ``with cd`` is not fully implemented for the remote side of things, but we expect it will be soon. For now we fall back to command chaining with ``&&``. And, notably, now that we care about selecting host targets, we refer to our earlier definition of a default host list -- ``my_hosts`` -- when declaring the default host list for this task. :: @task(hosts=my_hosts) def deploy(c): code_dir = "/srv/django/myproject" if not c.run("test -d {}".format(code_dir), warn=True): cmd = "git clone user@vcshost:/path/to/repo/.git {}" c.run(cmd.format(code_dir)) c.run("cd {} && git pull".format(code_dir)) c.run("cd {} && touch app.wsgi".format(code_dir)) The whole thing --------------- Now we have the entire, upgraded fabfile that will work with modern Fabric:: from invoke import Exit from invocations.console import confirm from fabric import task my_hosts = ["my-server"] @task def test(c): result = c.local("./manage.py test my_app", warn=True) if not result and not confirm("Tests failed. Continue anyway?"): raise Exit("Aborting at user request.") @task def commit(c): c.local("git add -p && git commit") @task def push(c): c.local("git push") @task def prepare_deploy(c): test(c) commit(c) push(c) @task(hosts=my_hosts) def deploy(c): code_dir = "/srv/django/myproject" if not c.run("test -d {}".format(code_dir), warn=True): cmd = "git clone user@vcshost:/path/to/repo/.git {}" c.run(cmd.format(code_dir)) c.run("cd {} && git pull".format(code_dir)) c.run("cd {} && touch app.wsgi".format(code_dir))