Sourcegraph is the standard developer platform for code search and navigation at many of the largest and most advanced technology companies. With Sourcegraph, every company has access to the same kind of tools that Google and Facebook developers use every day.

Deploy or upgrade: Local | AWS | DigitalOcean | Kubernetes cluster

Introducing code-aware structural search

View on Vimeo

Sourcegraph 3.11 introduces structural search, a code-aware search syntax that can identify structural patterns in code. This enables advanced code search to specifically match patterns inside code structures such as function parameters and loop bodies.

It can be awkward or difficult to match code blocks or nested expressions with regexp. To meet this challenge we’ve introduced a new and easier way to search code that operates more closely on the parse tree of the input using Comby syntax for structural matching.

For example, this can be used to find all the places that fmt.Sprintf are unnecessary:

patterntype:structural 'fmt.Sprintf(":[str]")'

Or to identify all the places an http.Client is created with a Transport:

patterntype:structural "http.Client{:[a] Transport: :[b] :[c]}"

In these examples, the :[str] part is a hole with a descriptive name str that matches code. The important part is that these patterns understand that the structural pieces (e.g. parentheses and braces) are balanced.

Known limitations:

• Only indexed repositories will show results for structural search. On Sourcegraph, we index approximately 10,000 of the most popular repositories on GitHub.
• To use this query syntax you must include patterntype:structural to activate the search type.
• See additional functionality and limitations

Management console removed to simplify configuration

The management console has been completely removed and there is now a single site configuration on the Site admin -> Configuration page. The current management console configuration will be automatically merged into your site configuration after upgrading your instance.

In the rare event that your Sourcegraph instance is misconfigured and causes the UI to become inaccessible, we have added an escape hatch file that allows you to edit your site configuration without the web UI. Upon saving, the updated configuration will be synchronized from the file and applied to your Sourcegraph instance.

High-level health monitoring

Recent Sourcegraph versions introduced standardized Prometheus and Grafana monitoring bundled with Sourcegraph. Sourcegraph 3.11 introduces a new set of dashboards and high-level health metrics that make it easier to understand the health of your Sourcegraph instance at a glance:

These dashboards are built using a new set of combinatorial alerting metrics we have introduced for each service. These allow site admins to measure the number of critical and warning-class alerts their Sourcegraph instance is facing:

Alerting can be configured easily through these metrics, so admins can get Email, Slack, PagerDuty, (and many more) alerts when an instance is unhealthy.

Future versions will add more exhaustive alert definitions and more detailed information on these dashboards, so please stay tuned!

Global settings now configurable via local file

Some teams prefer updating Sourcegraph configuration in version control rather than through the Site admin UI. In Sourcegraph 3.11, admins can configure their instance to load global settings from a file on-disk using the new GLOBAL_SETTINGS_FILE environment variable. Sourcegraph 3.4 enabled this functionality for external services and site configuration.

Find and remove leaked NPM credentials

View on Vimeo

Credential leaks compromise organization code and are a top concern for security teams and developers alike. Sourcegraph 3.11 introduces a new campaign type to find and remove leaked NPM credentials across all repositories at your organization. The leaked credentials campaign type allows you to preview the proposed changes, create pull requests on your code hosts, and track the PRs in a burndown chart to ensure they are merged.

Code change management campaigns are in private beta. Watch the campaigns screencasts to see what we have planned, and apply for early access to campaigns for your organization.

Search query language statistics

View on Vimeo

A new experimental feature has been added to show language statistics about your search query. Search result pages now have a Stats link to a visual breakdown of the languages that comprise the results for the query. This data is also available through our GraphQL API.

Language analysis is computationally expensive, so this feature is currently behind a feature flag. To enable, update your global, organization, or user settings to include { “experimentalFeatures”: { “searchStats”: true } }.

In other news

How Sourcegraph helped Lyft go from monolith to microservices

This month we published a new case study showing how Sourcegraph code search helped ensure production stability at Lyft during their monolith to microservices decomposition.

See you at GitLab Commit 2020 in San Francisco

Sourcegraph is sponsoring GitLab Commit in San Francisco this January. We’re looking forward to speaking with GitLab users about the importance of IDE quality code intelligence when browsing and reviewing code on GitLab. We’d love to see you there! Come say hello and pick up some great swag at our booth on level 1.

Changelog

3.11.0

Important: If you use SITE_CONFIG_FILE or CRITICAL_CONFIG_FILE, please be sure to follow the steps in: migration notes for Sourcegraph v3.11+ after upgrading.

Added

• Language statistics by commit are available via the API. #6737
• Added a new page that shows language statistics for the results of a search query.
• Global settings can be configured from a local file using the environment variable GLOBAL_SETTINGS_FILE.
• High-level health metrics and dashboards have been added to Sourcegraph's monitoring (found under the Site admin -> Monitoring area). #7216
• Logging for GraphQL API requests not issued by Sourcegraph is now much more verbose, allowing for easier debugging of problematic queries and where they originate from. #5706
• A new campaign type finds and removes leaked NPM credentials. #6893
• Campaigns can now be retried to create failed changesets due to ephemeral errors (e.g. network problems when creating a pull request on GitHub). #6718
• The initial release of structural code search.

Changed

• repohascommitafter: search filter uses a more efficient git command to determine inclusion. #6739
• NODE_NAME can be specified instead of HOSTNAME for zoekt-indexserver. HOSTNAME was a confusing configuration to use in Pure-Docker Sourcegraph deployments. #6846
• The feedback toast now requests feedback every 60 days of usage (was previously only once on the 3rd day of use). #7165
• The lsif-server container now only has a dependency on Postgres, whereas before it also relied on Redis. #6880
• Renamed the GraphQL API LanguageStatistics fields to name, totalBytes, and totalLines (previously the field names started with an uppercase letter, which was inconsistent).
• Detecting a file's language uses a more accurate but slower algorithm. To revert to the old (faster and less accurate) algorithm, set the USE_ENHANCED_LANGUAGE_DETECTION env var to the string false (on the sourcegraph/server container, or if using the cluster deployment, on the sourcegraph-frontend pod).
• Diff and commit searches that make use of before: and after: filters to narrow their search area are now no longer subject to the 50-repository limit. This allows for creating saved searches on more than 50 repositories as before. #7215

Fixed

• Changes to external service configurations are reflected much faster. #6058
• Deleting an external service will not show warnings for the non-existent service. #5617
• Suggested search filter chips are quoted if necessary. #6498
• Remove potential panic in gitserver if heavily loaded. #6710
• Multiple fixes to make the preview and creation of campaigns more robust and a smoother user experience. #6682 #6625 #6658 #7088 #6766 #6717 #6659
• Repositories referenced in campaigns that are removed in an external service configuration change won't lead to problems with the syncing process anymore. #7015
• The Searcher dashboard (and the src_graphql_search_response Prometheus metric) now properly account for search alerts instead of them being incorrectly added to the timeout category. #7214
• In the experimental search pagination API, the cloning, missing, and other repository fields now return a well-defined set of results. #6000

Removed

• The management console has been removed. All critical configuration previously stored in the management console will be automatically migrated to your site configuration. For more information about this change, or if you use SITE_CONFIG_FILE / CRITICAL_CONFIG_FILE, please see the migration notes for Sourcegraph v3.11+.

The changelog for this and previous releases is available on GitHub.

Thank you

Thank you to the many people who contributed to Sourcegraph since the last release!

• @Antoine98
• @terinjokes
• @dbentley
• @zapp42
• @zurp
• @saurabh-hirani
• @Warchant
• @sfllaw
• @mipearson

Deploy or upgrade: Local | AWS | DigitalOcean | Kubernetes cluster

From the entire @sourcegraph team, happy coding!