メインコンテンツまでスキップ

失敗の集約

RSpec::Expectationsは、aggregate_failuresを提供しており、このAPIを使用すると、最初の失敗で中断するのではなく、一度にすべての失敗をグループ化して表示することができます。 RSpec::Coreは、この機能をいくつかの方法で改善しています。

  • RSpec::Coreは、コードスニペットとバックトレースをサブ失敗に追加することで、失敗の出力を大幅に向上させます。
  • RSpec::Coreは、この機能のためのメタデータの統合を提供します。:aggregate_failuresとタグ付けされた各例は、aggregate_failuresブロックでラップされます。また、config.define_derived_metadataを使用して、これをすべての例に自動的に適用することもできます。

メタデータ形式は非常に便利ですが、複数の異なるステップを持つエンドツーエンドテストには適していない場合があります。たとえば、(1)リクエストを行い、(2)リダイレクトを期待し、(3)リダイレクトに従い、(4)特定のレスポンスを期待するHTTPクライアントのワークフローの仕様を考えてみてください。もしexpect(response.status).to be_between(300, 399)の期待が失敗した場合、次のステップ(リダイレクトに従う)を実行することはできませんので、即座に中断したいと思うでしょう。このような状況では、テストワークフローの各ステップを表す期待をラップするために、aggregate_failuresブロック形式を使用することをお勧めします。

背景​

以下の内容で "lib/client.rb" という名前のファイルがあるとします:

Response = Struct.new(:status, :headers, :body)

class Client
  def self.make_request(url='/')
    Response.new(404, { "Content-Type" => "text/plain" }, "Not Found")
  end
end

aggregate_failuresブロック形式の使用​

以下の内容で "spec/use_block_form_spec.rb" という名前のファイルがあるとします:

require 'client'

RSpec.describe Client do
  after do
    # this should be appended to failure list
    expect(false).to be(true), "after hook failure"
  end

  around do |ex|
    ex.run
    # this should also be appended to failure list
    expect(false).to be(true), "around hook failure"
  end

  it "returns a successful response" do
    response = Client.make_request

    aggregate_failures "testing response" do
      expect(response.status).to eq(200)
      expect(response.headers).to include("Content-Type" => "application/json")
      expect(response.body).to eq('{"message":"Success"}')
    end
  end
end

rspec spec/use_block_form_spec.rb を実行すると

すべての失敗が表示されるはずです:

Failures:

  1) Client returns a successful response
     Got 3 failures:

     1.1) Got 3 failures from failure aggregation block "testing response".
          # ./spec/use_block_form_spec.rb:18
          # ./spec/use_block_form_spec.rb:10

          1.1.1) Failure/Error: expect(response.status).to eq(200)

                   expected: 200
                        got: 404

                   (compared using ==)
                 # ./spec/use_block_form_spec.rb:19

          1.1.2) Failure/Error: expect(response.headers).to include("Content-Type" => "application/json")
                   expected {"Content-Type" => "text/plain"} to include {"Content-Type" => "application/json"}
                   Diff:
                   @@ -1 +1 @@
                   -"Content-Type" => "application/json",
                   +"Content-Type" => "text/plain",
                 # ./spec/use_block_form_spec.rb:20

          1.1.3) Failure/Error: expect(response.body).to eq('{"message":"Success"}')

                   expected: "{\"message\":\"Success\"}"
                        got: "Not Found"

                   (compared using ==)
                 # ./spec/use_block_form_spec.rb:21

     1.2) Failure/Error: expect(false).to be(true), "after hook failure"
            after hook failure
          # ./spec/use_block_form_spec.rb:6
          # ./spec/use_block_form_spec.rb:10

     1.3) Failure/Error: expect(false).to be(true), "around hook failure"
            around hook failure
          # ./spec/use_block_form_spec.rb:12

aggregate_failuresブロック形式の使用​

以下の内容で "spec/use_block_form_spec.rb" という名前のファイルがあるとします:

require 'client'

RSpec.describe Client do
  after do
    # this should be appended to failure list
    expect(false).to be(true), "after hook failure"
  end

  around do |ex|
    ex.run
    # this should also be appended to failure list
    expect(false).to be(true), "around hook failure"
  end

  it "returns a successful response" do
    response = Client.make_request

    aggregate_failures "testing response" do
      expect(response.status).to eq(200)
      expect(response.headers).to include("Content-Type" => "application/json")
      expect(response.body).to eq('{"message":"Success"}')
    end
  end
end

rspec spec/use_block_form_spec.rb を実行すると

すべての失敗が表示されるはずです:

Failures:

  1) Client returns a successful response
     Got 3 failures:

     1.1) Got 3 failures from failure aggregation block "testing response".
          # ./spec/use_block_form_spec.rb:18
          # ./spec/use_block_form_spec.rb:10

          1.1.1) Failure/Error: expect(response.status).to eq(200)

                   expected: 200
                        got: 404

                   (compared using ==)
                 # ./spec/use_block_form_spec.rb:19

          1.1.2) Failure/Error: expect(response.headers).to include("Content-Type" => "application/json")
                   expected {"Content-Type" => "text/plain"} to include {"Content-Type" => "application/json"}
                   Diff:
                   @@ -1,2 +1,2 @@
                   -"Content-Type" => "application/json",
                   +"Content-Type" => "text/plain",
                 # ./spec/use_block_form_spec.rb:20

          1.1.3) Failure/Error: expect(response.body).to eq('{"message":"Success"}')

                   expected: "{\"message\":\"Success\"}"
                        got: "Not Found"

                   (compared using ==)
                 # ./spec/use_block_form_spec.rb:21

     1.2) Failure/Error: expect(false).to be(true), "after hook failure"
            after hook failure
          # ./spec/use_block_form_spec.rb:6
          # ./spec/use_block_form_spec.rb:10

     1.3) Failure/Error: expect(false).to be(true), "around hook failure"
            around hook failure
          # ./spec/use_block_form_spec.rb:12

:aggregate_failures メタデータを使用する​

Given "spec/use_metadata_spec.rb" という名前のファイルがある場合、以下の内容である:

require 'client'

RSpec.describe Client do
  it "follows a redirect", :aggregate_failures do
    response = Client.make_request

    expect(response.status).to eq(302)
    expect(response.body).to eq('{"message":"Redirect"}')

    redirect_response = Client.make_request(response.headers.fetch('Location'))

    expect(redirect_response.status).to eq(200)
    expect(redirect_response.body).to eq('{"message":"OK"}')
  end
end

When rspec spec/use_metadata_spec.rb を実行すると

Then 失敗し、すべての失敗がリストされる:

Failures:

  1) Client follows a redirect
     Got 2 failures and 1 other error:

     1.1) Failure/Error: expect(response.status).to eq(302)

            expected: 302
                 got: 404

            (compared using ==)
          # ./spec/use_metadata_spec.rb:7

     1.2) Failure/Error: expect(response.body).to eq('{"message":"Redirect"}')

            expected: "{\"message\":\"Redirect\"}"
                 got: "Not Found"

            (compared using ==)
          # ./spec/use_metadata_spec.rb:8

     1.3) Failure/Error: redirect_response = Client.make_request(response.headers.fetch('Location'))
          KeyError:
            key not found: "Location"
          # ./spec/use_metadata_spec.rb:10
          # ./spec/use_metadata_spec.rb:10

define_derived_metadata を使用してグローバルに失敗の集約を有効にする​

Given "spec/enable_globally_spec.rb" という名前のファイルがある場合、以下の内容である:

require 'client'

RSpec.configure do |c|
  c.define_derived_metadata do |meta|
    meta[:aggregate_failures] = true
  end
end

RSpec.describe Client do
  it "returns a successful response" do
    response = Client.make_request

    expect(response.status).to eq(200)
    expect(response.headers).to include("Content-Type" => "application/json")
    expect(response.body).to eq('{"message":"Success"}')
  end
end

When rspec spec/enable_globally_spec.rb を実行すると

Then 失敗し、すべての失敗がリストされる:

Failures:

  1) Client returns a successful response
     Got 3 failures:

     1.1) Failure/Error: expect(response.status).to eq(200)

            expected: 200
                 got: 404

            (compared using ==)
          # ./spec/enable_globally_spec.rb:13

     1.2) Failure/Error: expect(response.headers).to include("Content-Type" => "application/json")
            expected {"Content-Type" => "text/plain"} to include {"Content-Type" => "application/json"}
            Diff:
            @@ -1 +1 @@
            -"Content-Type" => "application/json",
            +"Content-Type" => "text/plain",
          # ./spec/enable_globally_spec.rb:14

     1.3) Failure/Error: expect(response.body).to eq('{"message":"Success"}')

            expected: "{\"message\":\"Success\"}"
                 got: "Not Found"

            (compared using ==)
          # ./spec/enable_globally_spec.rb:15

define_derived_metadata を使用してグローバルに失敗の集約を有効にする​

Given "spec/enable_globally_spec.rb" という名前のファイルがある場合、以下の内容である:

require 'client'

RSpec.configure do |c|
  c.define_derived_metadata do |meta|
    meta[:aggregate_failures] = true
  end
end

RSpec.describe Client do
  it "returns a successful response" do
    response = Client.make_request

    expect(response.status).to eq(200)
    expect(response.headers).to include("Content-Type" => "application/json")
    expect(response.body).to eq('{"message":"Success"}')
  end
end

When rspec spec/enable_globally_spec.rb を実行すると

Then 失敗し、すべての失敗がリストされる:

Failures:

  1) Client returns a successful response
     Got 3 failures:

     1.1) Failure/Error: expect(response.status).to eq(200)

            expected: 200
                 got: 404

            (compared using ==)
          # ./spec/enable_globally_spec.rb:13

     1.2) Failure/Error: expect(response.headers).to include("Content-Type" => "application/json")
            expected {"Content-Type" => "text/plain"} to include {"Content-Type" => "application/json"}
            Diff:
            @@ -1,2 +1,2 @@
            -"Content-Type" => "application/json",
            +"Content-Type" => "text/plain",
          # ./spec/enable_globally_spec.rb:14

     1.3) Failure/Error: expect(response.body).to eq('{"message":"Success"}')

            expected: "{\"message\":\"Success\"}"
                 got: "Not Found"

            (compared using ==)
          # ./spec/enable_globally_spec.rb:15

ネストされた失敗の集約が機能する​

Given "spec/nested_failure_aggregation_spec.rb" という名前のファイルがある場合、以下の内容である:

require 'client'

RSpec.describe Client do
  it "returns a successful response", :aggregate_failures do
    response = Client.make_request

    expect(response.status).to eq(200)

    aggregate_failures "testing headers" do
      expect(response.headers).to include("Content-Type" => "application/json")
      expect(response.headers).to include("Content-Length" => "21")
    end

    expect(response.body).to eq('{"message":"Success"}')
  end
end

When rspec spec/nested_failure_aggregation_spec.rb を実行すると

Then 失敗し、すべての失敗がリストされる:

Failures:

  1) Client returns a successful response
     Got 3 failures:

     1.1) Failure/Error: expect(response.status).to eq(200)

            expected: 200
                 got: 404

            (compared using ==)
          # ./spec/nested_failure_aggregation_spec.rb:7

     1.2) Got 2 failures from failure aggregation block "testing headers".
          # ./spec/nested_failure_aggregation_spec.rb:9

          1.2.1) Failure/Error: expect(response.headers).to include("Content-Type" => "application/json")
                   expected {"Content-Type" => "text/plain"} to include {"Content-Type" => "application/json"}
                   Diff:
                   @@ -1 +1 @@
                   -"Content-Type" => "application/json",
                   +"Content-Type" => "text/plain",
                 # ./spec/nested_failure_aggregation_spec.rb:10

          1.2.2) Failure/Error: expect(response.headers).to include("Content-Length" => "21")
                   expected {"Content-Type" => "text/plain"} to include {"Content-Length" => "21"}
                   Diff:
                   @@ -1 +1 @@
                   -"Content-Length" => "21",
                   +"Content-Type" => "text/plain",
                 # ./spec/nested_failure_aggregation_spec.rb:11

     1.3) Failure/Error: expect(response.body).to eq('{"message":"Success"}')

            expected: "{\"message\":\"Success\"}"
                 got: "Not Found"

            (compared using ==)
          # ./spec/nested_failure_aggregation_spec.rb:14

ネストされた失敗の集約が機能する​

Given "spec/nested_failure_aggregation_spec.rb" という名前のファイルがある場合、以下の内容である:

require 'client'

RSpec.describe Client do
  it "returns a successful response", :aggregate_failures do
    response = Client.make_request

    expect(response.status).to eq(200)

    aggregate_failures "testing headers" do
      expect(response.headers).to include("Content-Type" => "application/json")
      expect(response.headers).to include("Content-Length" => "21")
    end

    expect(response.body).to eq('{"message":"Success"}')
  end
end

When rspec spec/nested_failure_aggregation_spec.rb を実行すると

Then 失敗し、すべての失敗がリストされる:

Failures:

  1) Client returns a successful response
     Got 3 failures:

     1.1) Failure/Error: expect(response.status).to eq(200)

            expected: 200
                 got: 404

            (compared using ==)
          # ./spec/nested_failure_aggregation_spec.rb:7

     1.2) Got 2 failures from failure aggregation block "testing headers".
          # ./spec/nested_failure_aggregation_spec.rb:9

          1.2.1) Failure/Error: expect(response.headers).to include("Content-Type" => "application/json")
                   expected {"Content-Type" => "text/plain"} to include {"Content-Type" => "application/json"}
                   Diff:
                   @@ -1,2 +1,2 @@
                   -"Content-Type" => "application/json",
                   +"Content-Type" => "text/plain",
                 # ./spec/nested_failure_aggregation_spec.rb:10

          1.2.2) Failure/Error: expect(response.headers).to include("Content-Length" => "21")
                   expected {"Content-Type" => "text/plain"} to include {"Content-Length" => "21"}
                   Diff:
                   @@ -1,2 +1,2 @@
                   -"Content-Length" => "21",
                   +"Content-Type" => "text/plain",
                 # ./spec/nested_failure_aggregation_spec.rb:11

     1.3) Failure/Error: expect(response.body).to eq('{"message":"Success"}')

            expected: "{\"message\":\"Success\"}"
                 got: "Not Found"

            (compared using ==)
          # ./spec/nested_failure_aggregation_spec.rb:14

モックの期待失敗も集約される​

Given "spec/mock_expectation_failure_spec.rb" という名前のファイルがある場合、以下の内容である:

require 'client'

RSpec.describe "Aggregating Failures", :aggregate_failures do
  it "has a normal expectation failure and a message expectation failure" do
    client = double("Client")
    expect(client).to receive(:put).with("updated data")
    allow(client).to receive(:get).and_return(Response.new(404, {}, "Not Found"))

    response = client.get
    expect(response.status).to eq(200)
  end
end

When rspec spec/mock_expectation_failure_spec.rb を実行すると

Then 失敗し、すべての失敗がリストされる:

Failures:

  1) Aggregating Failures has a normal expectation failure and a message expectation failure
     Got 2 failures:

     1.1) Failure/Error: expect(response.status).to eq(200)

            expected: 200
                 got: 404

            (compared using ==)
          # ./spec/mock_expectation_failure_spec.rb:10

     1.2) Failure/Error: expect(client).to receive(:put).with("updated data")
            (Double "Client").put("updated data")
                expected: 1 time with arguments: ("updated data")
                received: 0 times
          # ./spec/mock_expectation_failure_spec.rb:6

保留中のテストは失敗の集約と正しく統合される​

Given "spec/pending_spec.rb" という名前のファイルがある場合、以下の内容である:

require 'client'

RSpec.describe Client do
  it "returns a successful response", :aggregate_failures do
    pending "Not yet ready"
    response = Client.make_request

    expect(response.status).to eq(200)
    expect(response.headers).to include("Content-Type" => "application/json")
    expect(response.body).to eq('{"message":"Success"}')
  end
end

When rspec spec/pending_spec.rb を実行すると

Then パスし、すべての保留中の例がリストされる:

Pending: (Failures listed here are expected and do not affect your suite's status)

  1) Client returns a successful response
     # Not yet ready
     Got 3 failures:

     1.1) Failure/Error: expect(response.status).to eq(200)

            expected: 200
                 got: 404

            (compared using ==)
          # ./spec/pending_spec.rb:8

     1.2) Failure/Error: expect(response.headers).to include("Content-Type" => "application/json")
            expected {"Content-Type" => "text/plain"} to include {"Content-Type" => "application/json"}
            Diff:
            @@ -1 +1 @@
            -"Content-Type" => "application/json",
            +"Content-Type" => "text/plain",
          # ./spec/pending_spec.rb:9

     1.3) Failure/Error: expect(response.body).to eq('{"message":"Success"}')

            expected: "{\"message\":\"Success\"}"
                 got: "Not Found"

            (compared using ==)
          # ./spec/pending_spec.rb:10

ペンディングは集約された失敗と正しく統合されます​

Given ファイル名が "spec/pending_spec.rb" のファイルがあるとき:

require 'client'

RSpec.describe Client do
  it "returns a successful response", :aggregate_failures do
    pending "Not yet ready"
    response = Client.make_request

    expect(response.status).to eq(200)
    expect(response.headers).to include("Content-Type" => "application/json")
    expect(response.body).to eq('{"message":"Success"}')
  end
end

When rspec spec/pending_spec.rb を実行すると

Then ペンディングの例がすべて表示され、パスするはずです:

Pending: (Failures listed here are expected and do not affect your suite's status)

  1) Client returns a successful response
     # Not yet ready
     Got 3 failures:

     1.1) Failure/Error: expect(response.status).to eq(200)

            expected: 200
                 got: 404

            (compared using ==)
          # ./spec/pending_spec.rb:8

     1.2) Failure/Error: expect(response.headers).to include("Content-Type" => "application/json")
            expected {"Content-Type" => "text/plain"} to include {"Content-Type" => "application/json"}
            Diff:
            @@ -1,2 +1,2 @@
            -"Content-Type" => "application/json",
            +"Content-Type" => "text/plain",
          # ./spec/pending_spec.rb:9

     1.3) Failure/Error: expect(response.body).to eq('{"message":"Success"}')

            expected: "{\"message\":\"Success\"}"
                 got: "Not Found"

            (compared using ==)
          # ./spec/pending_spec.rb:10