[Zurich] additional closure statuses work

Requirements:
=============

1: DM to be able to mark a problem as Wish, Extern, Not contactable,
Hidden, Jurisdiction unknown

2: SDM to be able to mark a problem as Not contactable.

3: DM to be able to add information to such a problem, and trigger
emails to external agencies, and to user, closing the problem at the
same time.

4: Closed reports must be categorized as Wish/Extern/NC/Hidden/JU for
statistical reporting

5: Non-closed reports must appear as open, to allow DM to see them in
the list, similar to what is currently done for Ruckmeldung Ausstehend
(Feedback outstanding).

5b: however "Hidden" problems must be immediately hidden from public
view.

Proposal:
=========

a) Requirement 1 will be carried out in exactly the same way by
DM.  e.g. Category dropdown select

b) Requirement 2 will be carried out in exactly the same way by SDM.
e.g. "Not contactable" button

c) BUT, for a), c), the category will be diverted to "Ruckmeldung
Ausstehend" (Feedback outstanding), and metadata will be saved with the
appropriate closure type.

d) NOTE: this means the report will still be visible to public as "In
bearbeitung" (Except for Hidden reports, as per 5b)

e) The extra fields will be presented on the "Ruckmeldung Ausstehend"
page, just as defined in the PDF, depending on the metadata.  e.g.
    - Wish: Zustaendige Stelle + Message
    - Extern: External body + Message2
    - Not contactable: (no extra info)
    - Hidden: (no extra info)
    - Jurisdiction unknown: (no extra info)

f) When the DM publishes the answer (e.g. "Antwort Veroeffentlichen"
button), the state will normally change to Fixed. BUT in the new cases,
the metadata saved in step c) will be checked, and the report will
change state to Wish/Extern/Not contactable/Hidden/Jurisdiction unknown,
as appropriate.  The report will now be visible/hidden as previously
discussed depending on the new state.

g) If the report is edited again (in /admin/report_edit) it will be in
the relevant state, and any extra fields will be visible as per e).
Those fields may be updated.
Alternatively, the state may be changed, possibly triggering a) again,
and clearing any metadata.

 - Fix javascript interactions 690
 - show new statuses in summary
 - fix photos on SDM page
 - fix Planned => Planned+metadata transition
 - stop "assignation .select" js affecting templates
 - fix redirection for Extern and generally

See mysociety/FixMyStreet-Commercial#672, mysociety/FixMyStreet-Commercial#690

1 files changed, 112 insertions, 96 deletions

diff --git a/t/cobrand/zurich.t b/t/cobrand/zurich.t
index 038a2fe35..0a622cb2b 100644
--- a/t/cobrand/zurich.t
+++ b/t/cobrand/zurich.t
@@ -46,6 +46,7 @@ sub reset_report_state {
     $report->unset_extra_metadata('moderated_overdue');
     $report->unset_extra_metadata('subdiv_overdue');
     $report->unset_extra_metadata('closed_overdue');
+    $report->unset_extra_metadata('closure_status');
     $report->whensent(undef);
     $report->state('unconfirmed');
     $report->created($created) if $created;
@@ -122,7 +123,8 @@ FixMyStreet::override_config {
 }, sub {
     $mech->get_ok( '/report/' . $report->id );
 };
-$mech->content_contains('Überprüfung ausstehend');
+$mech->content_contains('Überprüfung ausstehend')
+    or die $mech->content;
 
 # Check logging in to deal with this report
 FixMyStreet::override_config {
@@ -215,102 +217,100 @@ sub get_moderated_count {
 
 subtest "report_edit" => sub {
 
-    reset_report_state($report);
-
-    ok ( ! $report->get_extra_metadata('moderated_overdue'), 'Report currently unmoderated' );
-
-    is get_moderated_count(), 0;
-
     FixMyStreet::override_config {
         ALLOWED_COBRANDS => [ 'zurich' ],
     }, sub {
+
+        reset_report_state($report);
+        ok ( ! $report->get_extra_metadata('moderated_overdue'), 'Report currently unmoderated' );
+        is get_moderated_count(), 0;
+
         $mech->get_ok( '/admin/report_edit/' . $report->id );
         $mech->content_contains( 'Unbestätigt' ); # Unconfirmed email
         $mech->submit_form_ok( { with_fields => { state => 'confirmed' } } );
         $mech->get_ok( '/report/' . $report->id );
 
         $report->discard_changes();
-        is $report->state, 'confirmed', 'state has been updated to confirmed';
-    };
-
-    $mech->content_contains('Aufgenommen');
-    $mech->content_contains('Test Test');
-    $mech->content_lacks('photo/' . $report->id . '.0.jpeg');
-    $mech->email_count_is(0);
 
-    $report->discard_changes;
+        $mech->content_contains('Aufgenommen');
+        $mech->content_contains('Test Test');
+        $mech->content_lacks('photo/' . $report->id . '.0.jpeg');
+        $mech->email_count_is(0);
 
-    is ( $report->get_extra_metadata('moderated_overdue'), 0, 'Report now marked moderated' );
-    is get_moderated_count(), 1;
+        $report->discard_changes;
 
+        is ( $report->get_extra_metadata('moderated_overdue'), 0, 'Report now marked moderated' );
+        is get_moderated_count(), 1;
 
-    # Set state back to 10 days ago so that report is overdue
-    my $created = $report->created;
-    reset_report_state($report, $created->clone->subtract(days => 10));
+        # Set state back to 10 days ago so that report is overdue
+        my $created = $report->created;
+        reset_report_state($report, $created->clone->subtract(days => 10));
 
-    is get_moderated_count(), 0;
+        is get_moderated_count(), 0;
 
-    FixMyStreet::override_config {
-        ALLOWED_COBRANDS => [ 'zurich' ],
-    }, sub {
         $mech->get_ok( '/admin/report_edit/' . $report->id );
         $mech->submit_form_ok( { with_fields => { state => 'confirmed' } } );
         $mech->get_ok( '/report/' . $report->id );
-    };
-    $report->discard_changes;
-    is ( $report->get_extra_metadata('moderated_overdue'), 1, 'moderated_overdue set correctly when overdue' );
-    is get_moderated_count(), 0, 'Moderated count not increased when overdue';
 
-    reset_report_state($report, $created);
+        $report->discard_changes;
+        is ( $report->get_extra_metadata('moderated_overdue'), 1, 'moderated_overdue set correctly when overdue' );
+        is get_moderated_count(), 0, 'Moderated count not increased when overdue';
+
+        reset_report_state($report, $created);
 
-    FixMyStreet::override_config {
-        ALLOWED_COBRANDS => [ 'zurich' ],
-    }, sub {
         $mech->get_ok( '/admin/report_edit/' . $report->id );
         $mech->submit_form_ok( { with_fields => { state => 'confirmed' } } );
         $mech->get_ok( '/report/' . $report->id );
-    };
-    $report->discard_changes;
-    is ( $report->get_extra_metadata('moderated_overdue'), 0, 'Marking confirmed sets moderated_overdue' );
-    is ( $report->get_extra_metadata('closed_overdue'), undef, 'Marking confirmed does NOT set closed_overdue' );
-    is get_moderated_count(), 1;
+        $report->discard_changes;
+        is ( $report->get_extra_metadata('moderated_overdue'), 0, 'Marking confirmed sets moderated_overdue' );
+        is ( $report->get_extra_metadata('closed_overdue'), undef, 'Marking confirmed does NOT set closed_overdue' );
+        is get_moderated_count(), 1;
 
-    FixMyStreet::override_config {
-        ALLOWED_COBRANDS => [ 'zurich' ],
-    }, sub {
         $mech->get_ok( '/admin/report_edit/' . $report->id );
         $mech->submit_form_ok( { with_fields => { state => 'hidden' } } );
         $mech->get_ok( '/admin/report_edit/' . $report->id );
-    };
-    $report->discard_changes;
-    is ( $report->get_extra_metadata('moderated_overdue'), 0, 'Still marked moderated_overdue' );
-    is ( $report->get_extra_metadata('closed_overdue'),    0, 'Marking hidden also set closed_overdue' );
-    is get_moderated_count(), 1, 'Check still counted moderated'
-        or diag $report->get_column('extra');
 
-    reset_report_state($report);
+        $report->discard_changes;
+        is ( $report->get_extra_metadata('moderated_overdue'), 0, 'Still marked moderated_overdue' );
+        is ( $report->get_extra_metadata('closed_overdue'),    undef, "Marking hidden doesn't set closed_overdue..." );
+        is ( $report->state, 'planned', 'Marking hidden actually sets state to planned');
+        is ( $report->get_extra_metadata('closure_status'), 'hidden', 'Marking hidden sets closure_status to hidden');
+        is get_moderated_count(), 1, 'Check still counted moderated'
+            or diag $report->get_column('extra');
+
+        # publishing actually sets hidden
+        $mech->form_with_fields( 'status_update' );
+        $mech->submit_form_ok( { button => 'publish_response' } );
+        $mech->get_ok( '/admin/report_edit/' . $report->id );
+        $report->discard_changes;
 
-    is ( $report->get_extra_metadata('moderated_overdue'), undef, 'Sanity check' );
-    is get_moderated_count(), 0;
+        is ( $report->get_extra_metadata('closed_overdue'),    0, "Closing as hidden sets closed_overdue..." );
+        is ( $report->state, 'hidden', 'Closing as hidden sets state to hidden');
+        is ( $report->get_extra_metadata('closure_status'), undef, 'Closing as hidden unsets closure_status');
 
-    # Check that setting to 'hidden' also triggers moderation
-    FixMyStreet::override_config {
-        ALLOWED_COBRANDS => [ 'zurich' ],
-    }, sub {
+
+        reset_report_state($report);
+        is ( $report->get_extra_metadata('moderated_overdue'), undef, 'Sanity check' );
+        is get_moderated_count(), 0;
+
+        # Check that setting to 'hidden' also triggers moderation
         $mech->get_ok( '/admin/report_edit/' . $report->id );
         $mech->submit_form_ok( { with_fields => { state => 'hidden' } } );
         $mech->get_ok( '/admin/report_edit/' . $report->id );
-    };
-    $report->discard_changes;
-    is ( $report->get_extra_metadata('moderated_overdue'), 0, 'Marking hidden from scratch sets moderated_overdue' );
-    is ( $report->get_extra_metadata('closed_overdue'),    0, 'Marking hidden from scratch also set closed_overdue' );
-    is get_moderated_count(), 1;
+        $mech->form_with_fields( 'status_update' );
+        $mech->submit_form_ok( { button => 'publish_response' } );
+
+        $report->discard_changes;
+        is ( $report->get_extra_metadata('moderated_overdue'), 0, 'Marking hidden from scratch sets moderated_overdue' );
+        is ( $report->get_extra_metadata('closed_overdue'),    0, 'Marking hidden from scratch also set closed_overdue' );
+        is get_moderated_count(), 1;
 
-    is ($cobrand->get_or_check_overdue($report), 0, 'sanity check');
-    $report->update({ created => $created->clone->subtract(days => 10) });
-    is ($cobrand->get_or_check_overdue($report), 0, 'overdue call not increased');
+        is ($cobrand->get_or_check_overdue($report), 0, 'sanity check');
+        $report->update({ created => $created->clone->subtract(days => 10) });
+        is ($cobrand->get_or_check_overdue($report), 0, 'overdue call not increased');
 
-    reset_report_state($report, $created);
+        reset_report_state($report, $created);
+    }
 };
 
 FixMyStreet::override_config {
@@ -335,7 +335,7 @@ FixMyStreet::override_config {
     $mech->content_contains( 'Originaltext: “Test Test 1 for ' . $division->id . ' Detail”' );
 
     $mech->get_ok( '/admin/report_edit/' . $report->id );
-    $mech->submit_form_ok( { with_fields => { body_subdivision => $subdivision->id, send_rejected_email => 1 } } );
+    $mech->submit_form_ok( { with_fields => { body_subdivision => $subdivision->id } } );
 
     $mech->get_ok( '/report/' . $report->id );
     $mech->content_contains('In Bearbeitung');
@@ -421,13 +421,12 @@ subtest 'SDM' => sub {
             $mech->get_ok( '/admin/report_edit/' . $report->id );
             $mech->submit_form_ok( { button => 'not_contactable', form_number => 2 } );
             $report->discard_changes;
-            is $report->state, 'partial', 'Report sent back to partial (not_contactable) state';
+            is $report->state, 'planned', 'Report sent back to Rueckmeldung ausstehend state';
+            is $report->get_extra_metadata('closure_status'), 'partial', 'Report sent back to partial (not_contactable) state';
             is $report->bodies_str, $division->id, 'Report sent back to division';
         };
     };
 
-    $report->update({ state => 'planned' });
-
     $mech->log_out_ok;
 };
 
@@ -438,6 +437,9 @@ FixMyStreet::override_config {
     $mech->get_ok( '/admin' );
 };
 
+reset_report_state($report);
+$report->update({ state => 'planned' });
+
 $mech->content_contains( 'report_edit/' . $report->id );
 $mech->content_contains( DateTime->now->strftime("%d.%m.%Y") );
 
@@ -466,7 +468,7 @@ like $email->header('From'), qr/division\@example.org/, 'from line looks correct
 like $email->body, qr/FINAL UPDATE/, 'body looks correct';
 $mech->clear_emails_ok;
 
-# Assign directly to planned, don't confirm email
+# Assign planned (via confirmed), don't confirm email
 @reports = $mech->create_problems_for_body( 1, $division->id, 'Second', {
     state              => 'unconfirmed',
     confirmed          => undef,
@@ -479,6 +481,8 @@ FixMyStreet::override_config {
     ALLOWED_COBRANDS => [ 'zurich' ],
 }, sub {
     $mech->get_ok( '/admin/report_edit/' . $report->id );
+    $mech->submit_form_ok( { with_fields => { state => 'confirmed' } } );
+    $mech->get_ok( '/admin/report_edit/' . $report->id );
     $mech->submit_form_ok( { with_fields => { state => 'planned' } } );
     $mech->get_ok( '/report/' . $report->id );
 };
@@ -490,6 +494,8 @@ FixMyStreet::override_config {
 }, sub {
     $mech->get_ok( '/admin/report_edit/' . $report->id );
     $mech->content_contains( 'Unbestätigt' );
+    $report->discard_changes;
+    $mech->form_with_fields( 'status_update' );
     $mech->submit_form_ok( { button => 'publish_response', with_fields => { status_update => 'FINAL UPDATE' } } );
 
     $mech->get_ok( '/report/' . $report->id );
@@ -515,18 +521,28 @@ subtest "external report triggers email" => sub {
     FixMyStreet::override_config {
         ALLOWED_COBRANDS => [ 'zurich' ],
     }, sub {
+
+        # required to see body_external field
+        $report->state('planned');
+        $report->set_extra_metadata('closure_status' => 'closed');
         # Set the public_response manually here because the default one will have line breaks that get escaped as HTML, causing the comparison to fail.
         $report->set_extra_metadata('public_response' => 'Freundliche Gruesse Ihre Stadt Zuerich');
-        $report->update({ state => 'closed' }); # required to see body_external field
+        $report->update;
+
         $mech->get_ok( '/admin/report_edit/' . $report->id );
+        $mech->form_with_fields( 'publish_response' );
         $mech->submit_form_ok( {
+            button => 'publish_response',
             with_fields => {
                 body_external => $external_body->id,
                 external_message => $EXTERNAL_MESSAGE,
             } });
+        $report->discard_changes;
         $mech->get_ok( '/report/' . $report->id );
     };
-    $mech->content_contains('Extern');
+    is ($report->state, 'closed', 'Report was closed correctly');
+    $mech->content_contains('Extern')
+        or die $mech->content;
     $mech->content_contains('Third Test');
     $mech->content_contains($report->get_extra_metadata('public_response')) or die $mech->content;
     send_reports_for_zurich();
@@ -543,13 +559,24 @@ subtest "external report triggers email" => sub {
             ALLOWED_COBRANDS => [ 'zurich' ],
         }, sub {
             $mech->get_ok( '/admin' );
-            $report->update({ state => 'closed' }); # required to see body_external field
+            # required to see body_external field
+            $report->state('planned');
+            $report->set_extra_metadata('closure_status' => 'closed');
+            $report->set_extra_metadata('public_response' => 'Freundliche Gruesse Ihre Stadt Zuerich');
+            $report->update;
+
             is $mech->uri->path, '/admin', "am logged in";
             $mech->content_contains( 'report_edit/' . $report->id );
             $mech->get_ok( '/admin/report_edit/' . $report->id );
-            $mech->submit_form_ok( { with_fields => { body_external => $external_body->id, third_personal => 1 } } );
-            $mech->get_ok( '/report/' . $report->id );
-        };
+            $mech->form_with_fields( 'publish_response' );
+            $mech->submit_form_ok( {
+                button => 'publish_response',
+                with_fields => {
+                    body_external => $external_body->id,
+                    third_personal => 1,
+                } });
+                $mech->get_ok( '/report/' . $report->id );
+            };
         $mech->content_contains('Extern');
         $mech->content_contains('Third Test');
         $mech->content_contains($report->get_extra_metadata('public_response'));
@@ -566,9 +593,18 @@ subtest "external report triggers email" => sub {
         FixMyStreet::override_config {
             ALLOWED_COBRANDS => [ 'zurich' ],
         }, sub {
-            $report->update({ state => 'investigating' }); # Wish
+            # set as wish
+            $report->discard_changes;
+            $report->state('planned');
+            $report->set_extra_metadata('closure_status' => 'investigating');
+            $report->update;
+            is ($report->state, 'planned', 'Sanity check') or die;
+
             $mech->get_ok( '/admin/report_edit/' . $report->id );
+
+            $mech->form_with_fields( 'publish_response' );
             $mech->submit_form_ok( {
+                button => 'publish_response',
                 with_fields => {
                     body_external => $external_body->id,
                     external_message => $EXTERNAL_MESSAGE,
@@ -702,25 +738,6 @@ subtest "problems can't be assigned to deleted bodies" => sub {
     $mech->log_out_ok;
 };
 
-subtest "hidden report email are only sent when requested" => sub {
-    $user = $mech->log_in_ok( 'dm1@example.org') ;
-    $report->set_extra_metadata(email_confirmed => 1);
-    $report->update;
-    FixMyStreet::override_config {
-        ALLOWED_COBRANDS => [ 'zurich' ],
-    }, sub {
-        $mech->get_ok( '/admin/report_edit/' . $report->id );
-        $mech->submit_form_ok( { with_fields => { state => 'hidden', send_rejected_email => 1 } } );
-        $mech->email_count_is(1);
-        $mech->clear_emails_ok;
-        $mech->get_ok( '/admin/report_edit/' . $report->id );
-        $mech->submit_form_ok( { with_fields => { state => 'hidden', send_rejected_email => undef } } );
-        $mech->email_count_is(0);
-        $mech->clear_emails_ok;
-        $mech->log_out_ok;
-    };
-};
-
 subtest "photo must be supplied for categories that require it" => sub {
     FixMyStreet::App->model('DB::Contact')->find_or_create({
         body => $division,
@@ -774,7 +791,6 @@ subtest "test stats" => sub {
         if (defined $export_count) {
             is $export_count - $EXISTING_REPORT_COUNT, 3, 'Correct number of reports';
             $mech->content_contains('fixed - council');
-            $mech->content_contains(',hidden,');
         }
     };
 };
@@ -788,8 +804,8 @@ subtest "test admin_log" => sub {
 
     # XXX: following is dependent on all of test up till now, rewrite to explicitly
     # test which things need to be logged!
-    is scalar @entries, 1, 'State changes logged'; 
-    is $entries[-1]->action, 'state change to hidden', 'State change logged as expected';
+    is scalar @entries, 3, 'State changes logged'; 
+    is $entries[-1]->action, 'state change to investigating', 'State change logged as expected';
 };
 
 subtest 'email images to external partners' => sub {