Announcing Sourcegraph 3.0

Quinn Slack on February 8, 2019

Sourcegraph 3.0 is now shipping! This release comes with HUGE improvements so you can search and navigate all your organization's code more quickly, with code intelligence (go-to-definition and find-references) for all languages.

What is Sourcegraph? Sourcegraph is an open source, self-hosted, cross-repository code search and navigation tool, with an efficient web interface and "feels-like-native" integration into your code host. Install or upgrade Sourcegraph.

Highlights of this release:

Plus, there are several new and improved Sourcegraph extensions:

Also, in case you missed it, Sourcegraph is now open source (Apache-licensed).

Want to hear about new features and releases as soon as they're available? Follow @sourcegraph.

Ready to install or upgrade a self-hosted Sourcegraph instance? You can also try these features on Sourcegraph.com for open source code.

Highlights

Code intelligence for all languages

Two of the most common questions developers have while reading code are:

  • "Where is this thing defined?"
  • "What calls/uses this thing?"

You can now answer these questions easily across all of your organization's code, with Sourcegraph's improved code intelligence (go-to-definition and find-references) for N languages.

It works immediately for all code with zero configuration because it uses heuristics to provide fuzzy results by default. This is surprisingly helpful and fast, and it finds results in your other repositories, too.

Want precise results? Many of the N language extensions also can provide precise results, by building and analyzing the code like a compiler (using a language server). See each extension's page (such as the TypeScript/JavaScript extension) for details.

As long as you're aware of the limitations of the default (non-language server) mode, you'll like it lot. For example, in this mode, if multiple functions in your code are named open, the reference results for open will include false positives. (If it bothers you, run the extension's associated language server for precise results, or disable the language extension.)

For the most popular languages, the new language extensions are enabled by default on Sourcegraph.com and self-hosted instances on Sourcegraph 3.0.

See it in action on Sourcegraph.com:

Basic code intelligence for all languages

Search query cheatsheet

We want anyone to be able to start using Sourcegraph for code search without reading a manual. That's why we made Sourcegraph search super fast and added one-click query suggestions (computed locally using heuristics) to refine your query.

But when you're searching more than a couple times daily, learning Sourcegraph's search keywords (such as repo:) will make you more productive. Click the new ? icon next to any search button to see a quick reference. See "Search query syntax" for more info.

Try it out (click the ? next to the search button):

Search query cheatsheet

More Sourcegraph extension APIs

We introduced new APIs for Sourcegraph extensions to add functionality to Sourcegraph and your code hosts. The full API is defined in sourcegraph.d.ts.

  • Progress notification (sourcegraph.app.activeWindow.withProgress(...) and showProgress(...)): extensions can show a window-level loading indicator for long-running tasks.
  • Light and dark color theme options for decorations (TextDocumentDecoration#light and TextDocumentDecoration#dark): when decorating code files with text and links, extensions can specify different colors to use depending on the prevailing UI theme (dark or light) to ensure sufficient contrast.
  • Text selection ranges (CodeEditor#selections): extensions can determine the currently selected range (or ranges, for multiple selections).
  • Workspace roots (sourcegraph.app.activeWindow.roots and onDidChangeRoots): extensions can determine the currently active repository and subscribe to changes (i.e, when the user navigates to a different repository or revision).
  • Custom panels (sourcegraph.app.createPanelView): extensions can define custom tabs in the panel, in addition to the standard definition and references tabs, such as for showing implementations, type definitions, and other information. The content can be a list of locations or arbitrary Markdown.
  • Streaming locations: providers (for references, definitions, and other kinds of locations) can now also return an observable, which lets them "stream" references in. Previously, they could only return an array or promise, so they were limited to a single result set.
  • Query transformers (sourcegraph.search.registerQueryTransformer): extensions can define query transformers to implement custom search keywords, such as transforming js.depends:react to a query matching JavaScript/TypeScript files that import the react package.
  • panel/toolbar menu contribution point: extensions can add actions that are shown in the panel (above references and definitions). This is useful for actions that affect the result set of references or definitions, such as changing which types of references are shown (e.g., function calls only vs. all uses).
  • Extension deactivation: this allows extensions to be cleanly unloaded when they are not in use, so that Sourcegraph runs smoothly for users.
  • Builtin commands: these perform useful actions in Sourcegraph, such as updating a user's settings and accessing the Sourcegraph GraphQL API.
  • searchFilters contribution: extensions can add static search filters to suggest additions to a user's query. (This is the first step toward extensions being able to provide search results, which will allow Sourcegraph to search more things, such as GitHub issues, logs, and configuration data.)

We shipped other improvements to the extension development process:

The Sourcegraph extensions authoring documentation also got some new tutorials:

Extension registry improvements

The Sourcegraph.com extension registry lists all publicly available Sourcegraph extensions. You can also access it on your own Sourcegraph instance at User menu > Extensions. Here's what's new:

Management console and site config improvements

The new management console is a separate, failsafe web interface for changing critical site configuration. This improves the experience for admins:

  • You'll never be locked out of your instance, even if you make a configuration mistake, because the management console is a separate service. (Previously, you could edit critical configuration online in the Sourcegraph site admin area, but a misconfiguration could make your instance inaccessible.)
  • Cluster deployments now support editing site configuration online. (Previously, you needed to edit Kubernetes .yml files and redeploy.)
  • The above two things also allow Sourcegraph to support automatic migrations of configuration, so you won't need to manually update site configuration when we add or change the configuration schema in releases.

The management console runs on a separate port (2633 by default) and is protected by strong, randomly generated password that is visible only in the site admin area:

Management console password is shown in site admin

Use this password to authenticate to https://sourcegraph.example.com:2633 (using your actual hostname instead). Be sure to use https (and note that it uses a self-signed certificate that you will need to trust on first use). Once you're in, you can edit critical configuration (with validation and autocompletion):

Management console

Deploy to more kinds of clusters

Now you have more options when you need to scale to a cluster of Sourcegraph. (Deploying Sourcegraph with Docker to a single container is the most popular deployment method, and we still recommend all instances start with it.)

The pure-Docker cluster deployment reference for Sourcegraph contains multiple Dockerfiles that use only the most basic orchestration primitives to connect to each other. You can adapt these files to deploy to your preferred cluster management/orchestration system, such as Netflix's Titus or Mesosphere. If you don't have a preferred system, we recommend deploying to Kubernetes (which was previously the only cluster deployment option).

See "Installing Sourcegraph on a cluster" for more information.

Authentication and repository permissions from GitHub

You can configure Sourcegraph to allow users to sign in via GitHub or GitHub Enterprise. (Many other user authentication providers are supported, such as SAML, OpenID Connect, GitLab, builtin username-password, and HTTP auth proxies).

Sourcegraph can also enforce GitHub or GitHub Enterprise repository access permissions, so that users can only search and view repositories on Sourcegraph if they are permitted to do so on GitHub. This feature makes Sourcegraph delegate authorization checks for repository access to GitHub, so your GitHub repository permissions are the source of truth.

See "GitHub integration with Sourcegraph" for more information.

Nginx-based HTTP server

All external HTTP serving and configuration (TLS, listen ports, Let's Encrypt, etc.) is now handled by Nginx, a popular, open source HTTP server. See "Sourcegraph nginx HTTP server settings" for information on how to configure the external web server for Sourcegraph.

Previously, Sourcegraph implemented these HTTP features internally. We found that most admins preferred to wrap Sourcegraph with Nginx (sometimes using ingress-nginx) anyway, so we decided to make that the preferred and only way. Now that Sourcegraph relies on Nginx, you have the full power and configurability of Nginx at your disposal.

Faster page load times

We trimmed the initial bundle size by ~50% for the Sourcegraph web app. This speeds up page loads on Sourcegraph.com and your own Sourcegraph instance. Thanks, Webpack! (As a JS Foundation member, Sourcegraph is proud to support Webpack.)

Datadog metrics extension

The new Datadog metrics extension adds View metric links at the end of lines of code that contain calls to record Datadog/statsd metrics. This makes it easy to see how code is behaving in production and in real time when you're reading and reviewing code.

Because this is a Sourcegraph extension, it works on Sourcegraph and on your code host (such as on GitHub code files and PRs).

For example, if your code records the time it takes to respond to queries, this extension makes it easy to jump to the query response time statistics from your code:

Line of code with Datadog metrics link

Documentation

The main docs.sourcegraph.com page was reorganized to make the most common documentation easier to get to. We've also added a lot more documentation pages throughout.

Engineering

This release includes more solid foundations for several Sourcegraph features:

  • Sourcegraph extensions are the sole source of code intelligence now. We removed the old way, which required a lot more Sourcegraph-specific custom integration work to add or configure a new language.
  • The management console makes it so that all configuration can be edited online and in-band, which means future Sourcegraph releases can auto-migrate their own config instead of requiring manual (and error-prone) steps by the site admin.
  • The new concept of external service connections will allow us to improve Sourcegraph's synchronization of repositories in a future release.
  • Standardizing on Nginx for serving HTTP lets us remove a lot of complex core code to support numerous use cases better handled by Nginx.

Thank you

Thank you to the many people who contributed to make Sourcegraph better since our last release!

Install or upgrade

Ready to install? Install Sourcegraph 3.0.1

Upgrading from 2.x or 3.0.0? See the migration guide

From the entire Sourcegraph team (@sourcegraph), happy coding!