Predefined rules#

When Protovalidate projects grow, the same custom rules or groups of standard rules often start to repeat. Just like you'd refactor repeated code into a function, predefined rules allow you to write these patterns once and reuse them across your project.

Example case#

It's common for a schema to need the same validation rules for several fields. This can get tediously repetitive:

person.proto

syntax = "proto3";

package bufbuild.people.v1;

import "buf/validate/validate.proto";

message Person {
  // given_name is required and must have between 1 and 50 characters.
  string given_name = 1 [
    (buf.validate.field).string.min_len = 1,
    (buf.validate.field).string.max_len = 50
  ];
  // middle_name is optional and, if present, must have between 1 and 50 characters.
  optional string middle_name = 2 [
    (buf.validate.field).string.min_len = 1,
    (buf.validate.field).string.max_len = 50
  ];
  // family_name is required and must have between 1 and 50 characters.
  string family_name = 3 [
    (buf.validate.field).string.min_len = 1,
    (buf.validate.field).string.max_len = 50
  ];
  // title is optional and, if present, must have between 1 and 25 characters.
  optional string title = 4 [
    (buf.validate.field).string.min_len = 1,
    (buf.validate.field).string.max_len = 25
  ];
  // suffix is optional and, if present, must have between 1 and 25 characters.
  optional string suffix = 5 [
    (buf.validate.field).string.min_len = 1,
    (buf.validate.field).string.max_len = 25
  ];
}

Instead of copying, pasting, and hoping for consistent maintenance, Protovalidate allows you to extend any standard rule message, defining common validation logic once and reusing it.

Creating predefined rules#

With predefined rules, you can create two rules to address this example:

A string rule for a "long name component" for given, middle, and family names.
A string rule for a "short name component" for titles and suffixes.

Extend a rule#

First, create a separate .proto file and extend a standard rule message. It's not required, but separating services, messages, and extensions and grouping predefined rules into files named after the message they extend is good practice.

Rules are extensions, so you must use either proto2 syntax or Protobuf Edition 2023. You're free to import and use them within proto3 files.

For the example above, create a predefined_string_rules.proto file to store all of your predefined string rules:

proto2Edition 2023

predefined_string_rules.proto

syntax = "proto2";

package bufbuild.people.v1;

import "buf/validate/validate.proto";

extend buf.validate.StringRules {}

predefined_string_rules.proto

edition = "2023";

package bufbuild.people.v1;

import "buf/validate/validate.proto";

extend buf.validate.StringRules {}

Because predefined rules are extensions, this file must use either proto2 syntax or Protobuf Edition 2023. You're free to import and use them within proto3 files.

Define rules#

Add a field to the extension for each predefined rule you want to create. Follow these guidelines:

The field type should match the type of value for your rule. Its value is accessible at runtime in CEL expressions as a variable named rule.
The field number must not conflict with any other extension of the same message across all Protobuf files in the project. See the Field numbers must be unique for more information.
The field must have an option of type buf.validate.predefined, which itself has a single cel field of type Rule. Its value is a custom CEL rule.

Following these guidelines, you can declare predefined long_name_component and short_name_component rules to cut down on the repetition:

proto2Edition 2023

predefined_string_rules.proto

extend buf.validate.StringRules {
  optional bool long_name_component = 81048952 [(buf.validate.predefined).cel = {
    id: "string.long_name_component"
    message: "value must have between 1 and 50 characters"
    expression: "this.size() > 0 && this.size() <= 50"
  }];
  optional bool short_name_component = 81048953 [(buf.validate.predefined).cel = {
    id: "string.long_name_component"
    message: "value must have between 1 and 25 characters"
    expression: "this.size() > 0 && this.size() <= 25"
  }];
}

predefined_string_rules.proto

extend buf.validate.StringRules {
  bool long_name_component = 81048952 [(buf.validate.predefined).cel = {
    id: "string.long_name_component"
    message: "value must have between 1 and 50 characters"
    expression: "this.size() > 0 && this.size() <= 50"
  }];
  bool short_name_component = 81048953 [(buf.validate.predefined).cel = {
    id: "string.long_name_component"
    message: "value must have between 1 and 25 characters"
    expression: "this.size() > 0 && this.size() <= 25"
  }];
}

Field numbers must be unique#

Extension numbers may be from 1000 to 536870911, inclusive, and must not conflict with any other extension to the same message. This restriction also applies to projects that consume Protobuf files indirectly as dependencies.

For private Protobuf schemas, use 100000 to 536870911. For public schemas, use 1000 to 99999 and register your extension with the Protobuf Global Extension Registry. This prevents conflicts when your schemas are used as dependencies.

Different kinds of rule can reuse the same extension number: 1000 in FloatRules is distinct from 1000 in Int32Rules.

Applying predefined rules#

Now that you've defined long_name_component and short_name_component rules, you can simplify the repetitive groups of standard rules in the Person message.

Be sure to import your rule file and surround the name of your extension with parentheses; extensions are always qualified by the package within which they're defined.

This example's predefined rules are in the same package as its messages. In other cases, usage must qualify the package name of the extension, like (buf.validate.field).float.(foo.bar.required_with_max)

proto3proto2Edition 2023

person.proto

syntax = "proto3";

package bufbuild.people.v1;

import "buf/validate/validate.proto";
import "bufbuild/people/v1/predefined_string_rules.proto";

message Person {
  string given_name = 1 [(buf.validate.field).string.(long_name_component) = true];
  optional string middle_name = 2 [(buf.validate.field).string.(long_name_component) = true];
  string family_name = 3 [(buf.validate.field).string.(long_name_component) = true];
  optional string title = 4 [(buf.validate.field).string.(short_name_component) = true];
  optional string suffix = 5 [(buf.validate.field).string.(short_name_component) = true];
}

person.proto

syntax = "proto2";

package bufbuild.people.v1;

import "buf/validate/validate.proto";
import "bufbuild/people/v1/predefined_string_rules.proto";

message Person {
  optional string given_name = 1 [
    (buf.validate.field).string.(long_name_component) = true,
    (buf.validate.field).required = true
  ];
  optional string middle_name = 2 [(buf.validate.field).string.(long_name_component) = true];
  optional string family_name = 3 [
    (buf.validate.field).string.(long_name_component) = true,
    (buf.validate.field).required = true
  ];
  optional string title = 4 [(buf.validate.field).string.(short_name_component) = true];
  optional string suffix = 5 [(buf.validate.field).string.(short_name_component) = true];
}

person.proto

edition = "2023";

package bufbuild.people.v1;

import "buf/validate/validate.proto";
import "bufbuild/people/v1/predefined_string_rules.proto";

message Person {
  string given_name = 1 [
    (buf.validate.field).string.(long_name_component) = true,
    (buf.validate.field).required = true
  ];
  string middle_name = 2 [(buf.validate.field).string.(long_name_component) = true];
  string family_name = 3 [
    (buf.validate.field).string.(long_name_component) = true,
    (buf.validate.field).required = true
  ];
  string title = 4 [(buf.validate.field).string.(short_name_component) = true];
  string suffix = 5 [(buf.validate.field).string.(short_name_component) = true];
}

Combining rules#

Predefined rules can be used in combination with any other rules. Extending the prior example, you could update person.proto to forbid family names from containing . using not_contains:

proto3proto2Edition 2023

Combining predefined rules with standard rules

string family_name = 3 [
    (buf.validate.field).string.(long_name_component) = true,
    (buf.validate.field).string.not_contains = "."
  ];

Combining predefined rules with standard rules

optional string family_name = 3 [
    (buf.validate.field).string.(long_name_component) = true,
    (buf.validate.field).string.not_contains = ".",
    (buf.validate.field).required = true
  ];

Combining predefined rules with standard rules

string family_name = 3 [
    (buf.validate.field).string.(long_name_component) = true,
    (buf.validate.field).string.not_contains = ".",
    (buf.validate.field).required = true
  ];

But you can also avoid repetition in person.proto with message literal syntax:

proto3proto2Edition 2023

Predefined rules with message literal syntax

string family_name = 3 [(buf.validate.field).string = {
    [long_name_component]: true
    not_contains: "."
  }];

Predefined rules with message literal syntax

optional string family_name = 3 [(buf.validate.field) = {
    string: {
      [long_name_component]: true
      not_contains: "."
    }
    required: true
  }];

Predefined rules with message literal syntax

string family_name = 3 [(buf.validate.field) = {
    string: {
      [long_name_component]: true
      not_contains: "."
    }
    required: true
  }];

Using rule values#

The prior example is a simple predefined rule: it doesn't use the rule's value (the boolean true) in its CEL expression. If the suffix field needed a unique maximum length like 40, someone might be tempted to stop using your predefined rules.

Referencing rule values in your CEL, you can create predefined rules that incorporate rule values into both their logic and validation messages.

Applying this to the prior example, you can create a single name_component rule in predefined_string_rules.proto that:

Uses the rule variable in its CEL expression to access a length value assigned to the rule (50, 40, or 25).
Returns an empty string to indicate the field's value is valid, and a dynamic error message when it's not.

Predefined rule using the rule value (proto2)

extend buf.validate.StringRules {
  optional int32 name_component = 80048954 [(buf.validate.predefined).cel = {
    id: "string.name_component"
    expression:
      "(this.size() > 0 && this.size() <= rule)"
      "? ''"
      ": 'value must have between 1 and ' + string(rule) + ' characters'"
  }];
}

Now person.proto is both simpler and more semantically expressive:

Declaring rule values (proto3)

message Person {
  string given_name = 1 [(buf.validate.field).string.(name_component) = 50];
  optional string middle_name = 2 [(buf.validate.field).string.(name_component) = 50];
  string family_name = 3 [(buf.validate.field).string.(name_component) = 50];
  optional string title = 4 [(buf.validate.field).string.(name_component) = 25];
  optional string suffix = 5 [(buf.validate.field).string.(name_component) = 40];
}

Resolving conflicts#

You can also use the rules variable in your CEL expression to resolve conflicts with other rules within the extended rule message¹.

Continuing the prior example, you could update the rule in predefined_string_rules.proto to delegate its minimum length check to the min_len rule, if present:

Predefined rule using the rules value (proto2)

    extend buf.validate.StringRules {
      optional int32 name_component = 80048954 [(buf.validate.predefined).cel = {
        id: "string.name_component"
        expression:
          "("
          "  (has(rules.min_len) ? true : this.size() > 0) && "
          "  this.size() <= rule"
          ")"
          "? ''"
          ": 'value must have between ' + "
          "  string(has(rules.min_len) ? rules.min_len : 1u) + ' and ' + "
          "  string(rule) + ' characters'"
      }];
    }

Now you can update the middle_name field to use min_len as an override:

Taking advantage of conflict resolution (proto3)

    optional string middle_name = 2 [(buf.validate.field).string = {
        [name_component]: 50
        min_len: 0
      }];

Learn more#

Now that you've mastered standard rules, custom rules, and predefined rules, it's time to put Protovalidate to work inside your RPC APIs or Kafka streams:

Add Protovalidate to Connect Go
Add Protovalidate to gRPC with quickstarts for gRPC and Go, gRPC and Java, or gRPC and Python.
Enforce Protovalidate rules in Kafka with Bufstream.

In the running example, this is an instance of the buf.validate.StringRules message extended by the predefined rule. ↩