This article will walk you through how to set up a custom AI property for Smart Import and AI Suggestions in the Repository.

A property is a structured metadata field that stores key information about workflows and records. Properties help you categorize, search, and report on contract data efficiently. Custom AI properties use AI to automatically extract unique contract details using Smart Import and Run AI. You can train Ironclad AI properties to improve their accuracy using examples.

Part 1: Prepare Data for a Custom AI Property

Before adding a Custom AI Property, you must ensure your data is structured for accurate AI extraction. For each new property you plan to add, do the following:

1. Verify Language: Currently, our system only supports English language extraction.
2. Verify the Data Type: Currently, our system supports all data types for custom properties. Below are two examples:
   1. Duration: Extract out a piece of text from the contract.
      • Example: License Grant Period
        • Clause Text: The license shall be granted for a period of 3 years.
        • Ground Truth: 3 years.
   2. Yes/No: This is the most complex type we support. It’s looking to see if the ground truth is in the clause of interest in your contract and if it is, we’ll display “Yes.” Currently, the AI model does not detect a “No” response.
      • Example: Is Breach Notice Required?
        • Clause Text: Vendor will notify BlueTruckCo of any breach of security that results in the accidental or unlawful destruction, loss, alteration, unauthorized disclosure of, or access to BlueTruckCo Data or Personal Data (a “Security Incident”) without undue delay after becoming aware of the Security Incident and, in any event, within 48 hours of becoming aware of such Security Incident.
        • Ground Truth: Vendor will notify BlueTruckCo of any breach of security.
3. Identify Table Compatibility: Custom AI Properties can be detected in Tables. For table detection, we only support a property showing up in a particular cell in a table, not aggregation across multiple table values.
4. Identify Clause Compatibility: Custom AI Properties can be detected in Clauses (every customer gets 10 free out of the box Custom AI Clauses). More information below on how to set these up for accurate detection. Ensure the property is associated with a clause. Custom AI Properties are detectable only within Ironclad provided AI Clauses or Custom AI Clauses. (Each customer receives 10 Custom AI Clauses by default.)
   1. Identify Property is in Clauses: Determine the AI Clauses where your property is found:
      • Example of Property in Clause: Breach Notice Period
      • Example of Property not in Clauses: 1PP vs. 3PP paper (since this doesn’t relate to a clause within a document)
   2. Identify Relevant Clauses for the Property: Determine the AI Clauses where your property is found. You can have a property detected on multiple, different AI Clauses.
      • Check AI Clauses from Ironclad.
      • Review any existing Custom AI Clauses.
      • Create a new Custom AI Clause incorporating paragraphs from the examples you found.
   3. Gather Training Examples: You’ll need diverse examples to train the AI effectively. Typically, 5–10 varied examples are sufficient for most properties. Diversity is key, not quantity. Include versions of the property you might encounter to improve detection accuracy.
      • To find examples, go to your records and look for the below. Keep records of both the paragraph where the property is located and the property itself. For existing properties, you can also add a column and sort by it to locate examples.
        • Your standard terms for that property.
        • Redlined versions you've seen in past negotiations.
        • Versions of the clause you've accepted from third-party contracts (if applicable).
      • Keep a record of the paragraph the property was found in and the property itself.

Part 2: Create a Custom AI Property

To create a Custom AI Property, you will navigate to Company Settings and use Data Manager to define your property. You can either create a new property, enable AI detection for an existing property, or customize an out-of-the-box property. After setting the property name and description, configure AI detection by specifying search boundaries, selecting whether or not the property shows up in clauses or tables, and providing training examples. Once configured, save and publish the property to enable AI-powered contract analysis for that property.

1. Click on your profile and then navigate to Company Settings > Data Manager. and click Create Property.

2. If you want to create a new property:

   1. Click Create Property.

   2. Add a Name that the users will see for the property in Repository.

   3. In the Type field, select a data type.

   4. Add Description with a description of the property that users internally can use to identify the property. This helps distinguish duplicates and ensures the correct property is used.

   5. Click Save. The property is created and a summary window displays on the right.

   6. Click Edit indicated by a pencil icon.

   7. Under Enable AI detection, click Enable. Continue to step 4.

      

3. If you want to enable AI for an existing property:

   1. Locate and click on the property you want to use. A summary window displays on the right.

   2. Click Edit indicated by a pencil icon.

   3. Under Enable AI detection, click Enable. Continue to step 4.

      

4. If you want to customize an out-of-the-box property:
   1. Locate and click on the out-of-the-box property you want to use. A summary window displays on the right.
   2. Click Edit indicated by a pencil icon.
   3. Under Enable AI detection, click Enable. Continue to step 5.

5. Fill in Property Name to be Sent to the LLM and Description to be Sent to the LLM. Both the fields will be used as part of the prompt sent to the LLM.

   1. When writing a description, aim for one concise paragraph. Include an explanation of what the property is, the context in which it appears, and specify the property type (like address, email, number, or amount). If you're aware of any common false positives or edge cases, briefly mention them to help improve accuracy. Don’t worry though about getting it perfectly written on the first go, you can iterate on this as you validate the property.

6. If the property shows up in a clause, in Search Boundaries, check the box next to Clauses and add the clauses where the Custom AI Property can be detected in Associated Clauses.

   1. Add Examples that will be used to train the AI. Clause should be the paragraph it’s found and Ground Truth the exact text in the Clause that is associated with your property. The Ground Truth will be normalized to whatever Data Type you selected. If you selected “Date” Data Type, then we’ll ensure the field detected is normalized to a date. If you selected “Yes/No” Data Type, then if we find the property, we’ll save “Yes” (currently, AI will never detect no).

      1. Example: Is Breach Notice Required?
         • Clause Text: Vendor will notify BlueTruckCo of any breach of security that results in the accidental or unlawful destruction, loss, alteration, unauthorized disclosure of, or access to BlueTruckCo Data or Personal Data (a “Security Incident”) without undue delay after becoming aware of the Security Incident and, in any event, within 48 hours of becoming aware of such Security Incident.
         • Ground Truth: Vendor will notify BlueTruckCo of any breach of security.

      

7. If the property shows up in a table, in Search Boundaries, check the box next to Tables.

8. Click Save & Publish

Part 3: Validate the Custom AI Property

Once the initial setup is done, verify how well the AI performs by testing it on contracts.

The best way to think about the process of training AI is to compare it to how you interact with ChatGPT. In this case, you update the examples and description of a Custom AI Property, upload some contracts, and then evaluate how well it performs. Similarly, with ChatGPT, you type a query, receive a response, and iterate on your query until you achieve the desired result.

1. Select Test Contracts: Choose 5 contracts, ideally that have the properties of interest already tagged on them and a mix of first-party and third-party documents if applicable.
2. Upload via Smart Import: Download these contracts and upload them through Smart Import to let the AI generate new predictions.
3. When a misclassification occurs (AI tags text as the wrong property):
   1. Update the AI property configuration description with this example to clarify what the property is not.
   2. Review the AI property configuration description for any adjustments that could make it more specific to avoid similar misclassifications.
4. When non-detection occurs (AI doesn’t tag the text as the property):
   1. Add the missed text as an example for that property.
   2. If the property appears in a table and table detection is off, add table detection.
   3. If the property appears in a new clause, include the clause where the property appeared as an example in one of the associated AI Clauses.
   4. Check the AI property configuration description to see if it could be refined for more precise detection.
   NOTE: After doing this exercise, if you still feel like the accuracy could be higher, try the following:

   • Revisit the AI property configuration description.
   • Review past contracts and add more variations of the property as examples.

Part 4: Tracking and Improving AI Accuracy

AI is an evolving tool, and it’s crucial to track accuracy as new contracts come in to improving performance over time.

1. Set up a tracking system (e.g., an Excel spreadsheet) with the following columns:
   • Clause Text
   • Predicted Property
   • Actual Property
   • Classification Issue (Misclassification or Non-detection)
2. During contract reviews, if a clause is:
   • Misclassified, log the example under “Misclassification.”
   • Not detected, log the example with its correct clause type under “Non-detection.”
3. Monthly review: Once a month, review the spreadsheet and update the Custom AI Properties using the same guidance as in Part 3.