Sourcegraph 3.5: Powerful new search filters, improved configuration, and Bitbucket Server repository permissions

Ryan Blunden and Christina Forney on June 20, 2019

Happy Pride Month 🏳️‍🌈 from all of us at Sourcegraph!

Sourcegraph is the standard developer platform for code search and navigation at many of the largest and most exacting technology companies. With Sourcegraph, every company has access to the same kind of tools that Google and Facebook developers use every day.

⏎ Multi-line search with newline (\n) characters

📅 Restrict search results to repositories with recent activity

📂 Restrict search to repositories with or without a specific filename

🔒 Introducing Bitbucket Server repository permissions
Enforce repository permissions defined in Bitbucket Server.

🌈 Improved code reviews with line decorations in pull requests
Use extensions like Codecov, Sentry, Datadog, and more to put info in context of a code review.

💾 Powering code alerts with saved searches
Improved UI to separate User and Organization saved searches.

🛠 New orgs field to optimize repository syncing for GitHub organizations
Easier configuration for cloning all repos in an organization.

💡 Repository syncing status for site admins
Experimental feature to convey the status of cloning repositories.

📝 Changelog

🎖️ Thank you

Deploy or upgrade: Local | AWS | DigitalOcean | Kubernetes cluster

Multi-line search with newline (\n) characters

Sourcegraph now supports multi-line searches by matching on \n, providing new search use cases. For example, to identify usage of Python decorators at the module and class level:

lang:python ^\s*@.*\n\s*def

Note: multi-line search is currently only supported on indexed repositories (and, inherently, default branches). We’re working to improve this.

Restrict search results to repositories with recent activity

When you’re searching over 10,000+ repositories in an enterprise setting, it’s common to get results from old repositories with no active maintainers. You can now exclude repositories with no recent commits (to their default branch) with the beta repohascommitafter search filter.

Examples:

  • Find Dockerfiles requiring upgrades from an obsolete version of Debian so you can contact the maintainers:

    file:Dockerfile FROM debian\:jessie|8 repohascommitafter:”1 month ago”

  • Find Terraform files with an aws_instance resource to analyze EBS volume configuration:

    aws_instance lang:hcl repohascommitafter:"1 month ago"

Note: repohascommitafter is considered beta as performance degrades significantly on very large repository sets. We are addressing this in 3.6.

Restrict search to repositories with or without a specific filename

Sometimes, you want to search repositories based on the existence or absence of a file. This is now possible with the new repohasfile search filter which comes in two forms, repohasfile and -repohasfile.

Examples:

  • Find applications with a Dockerfile, written in Go, that use Travis CI:

    repohasfile:Dockerfile file:\.travis.yml "language: go"

  • Find applications with a Dockerfile that do not contain a .dockerignore file:

    -repohasfile:\.dockerignore file:Dockerfile

  • Find Python applications using pytest without a pytest.ini or pytest.conf file:

    -repohasfile:(pytest\.ini|pytest\.conf) lang:python "import pytest"

Introducing Bitbucket Server repository permissions

Note: GitHub and GitLab repository permissions are already supported.

Configuring Sourcegraph to enforce Bitbucket Server repository permissions for code search and navigation is now available for instances with < 2,500 repositories.

Making this possible, is the new authorization field, which enables Sourcegraph to communicate with the Bitbucket Server through an application link. Check out the configuring Bitbucket Server and Sourcegraph guide for requirements and end to end configuration.

We are working closely with our customers who have larger Sourcegraph instances to improve performance beyond 10,000+ repositories.

Improved code reviews with line decorations in pull requests

View on Vimeo

Line decorations enhance code views with Sourcegraph extensions such as Codecov, Sentry, and Datadog. In 3.5, line decorations are now available on pull/merge requests on GitHub, Bitbucket Server, and GitLab.

Powering code alerts with saved searches

View on Vimeo

User-level and organization-level saved searches are now separate. You can view and manage saved searches in the user and organization profile areas, respectively.

Saved searches can be used to bookmark your frequently used searches. Additionally, you can monitor critical parts of your code by turning on saved search email notifications.

New orgs field to optimize repository syncing for GitHub organizations

For customers using GitHub with 1,000+ repositories, Sourcegraph sometimes exceeds GitHub’s search API rate limit during syncing, and may result in an incomplete set of cloned repositories.

To address this, a new orgs field has been added for customers who want to sync all repositories for their organization. This uses a different GitHub API that is not subject to rate limiting.

The orgs field is a list of GitHub organizations:

“orgs”: [
    “gorilla”
]

If filtering the list of repositories is required, e.g. archived:no forked:no the repositoryQuery should be used instead of orgs. If the result of a query entry in repositoryQuery exceeds 1,000 repositories, it will need to be split out over multiple entries. Please contact support if you require assistance.

Here is an example of the configuration to sync all repositories for the Gorilla organization:

3.5 org settings

This is the resulting set of repositories from the above configuration:

3.5 org repos

Repository syncing status for site admins

View on Vimeo

To give more visibility into the status of repository syncing and updating operations, we added an experimental status indicator in the navigation bar.

To enable the repository syncing status indicator, add this setting to your site configuration:

"experimentalFeatures": { "statusIndicator": "enabled" }

Due to the experimental status of this feature, Sourcegraph instances with 500+ repositories may experience confusing status updates.

3.5 changelog

Added

  • Indexed search now supports matching consecutive literal newlines, with queries like e.g. foo\nbar.* to search over multiple lines. #4138
  • The orgs setting in GitHub external service config allows admins to select all repositories from the specified organizations to be synced.
  • A new experimental search filter repohascommitafter:"30 days ago" allows users to exclude stale repositories that don't contain commits (to the branch being searched over) past a specified date from their search query.
  • The authorization setting in the Bitbucket Server external service config enables Sourcegraph to enforce the repository permissions defined in Bitbucket Server.
  • A new, experimental status indicator in the navigation bar allows admins to quickly see whether the configured repositories are up to date or how many are currently being updated in the background. You can enable the status indicator with the following site configuration: "experimentalFeatures": { "statusIndicator": "enabled" }.

Changed

  • The saved searches UI has changed. There is now a saved searches page in the user and organizations settings area. A saved search appears in the settings area of the user or organization it is associated with.

Fixed

  • Fixed repository search patterns which contain .*. Previously our optimizer would ignore .*, which in some cases would lead to our repository search excluding some repositories from the results.
  • Fixed an issue where the Phabricator native integration would be broken on recent Phabricator versions. This fix depends on v1.2 of the Phabricator extension.
  • Fixed an issue where the "Empty repository" banner would be shown on a repository page when starting to clone a repository.
  • Prevent data inconsistency on cached archives due to restarts. (#4366)
  • On the /extensions page, the UI is now less ambiguous when an extension has not been activated. (#4446)

See the full Sourcegraph changelog.

Thank you

Thank you to the many people who contributed to Sourcegraph since the last release!

Deploy or upgrade: Local | AWS | DigitalOcean | Kubernetes cluster

From the entire Sourcegraph team (@sourcegraph), happy coding!