Difference between revisions of "OCR service"

Revision as of 16:36, 27 March 2018 (view source) Dino.dewaen (talk | contribs) ← Older edit		Latest revision as of 07:53, 14 September 2021 (view source) Onea (talk | contribs) (→‎Available fields)
(35 intermediate revisions by one other user not shown)
Line 1:		Line 1:
		+	==General==
		+	<br/>{{info|The OCR serivce is only active between 8h untill 18h. Documents will only be validated during this time of the day.}}<br/>
		+	The OCR service is used for recognizing and validating documents coming from an external source. Your app can make use of it by sending PDF documents to Onea Transfer (documentation can be found [https://dev.onea.be/api/oneatransfer/index.html here]).<br/>
		+	Those documents will automatically be sent through our recognition server, where they will await validation by a manual validator.
−	== Web-service ==	+	<br/>{{info|The advantage of using a manual validator is we can increase the recognition percentage to a maximum.}}<br/>
−	For each request to the web-service, user credentials should be passed.	+	Schematic overview of the flow:
		+	<br/>[[image:app_ocr_integration.png|link=|800px]]<br/>
		+	<br/>
		+	# Your app is sending a PDF document to the OCR service
		+	# The OCR service sets the status to ''PENDING'' and sends a uuid back
		+	# The OCR service sets the status to ''PROCESSING'' when the document is successfully received and ready to process
		+	#: The OCR service also pokes the given callback URL saying there was a statusupdate
		+	# Your app gets the ''PROCESSING'' status from the OCR service
		+	# The OCR service sets the status to ''VALIDATION'' when the document is successfully processed and ready for manual validation
		+	#: The OCR service also pokes the given callback URL saying there was a statusupdate
		+	# Your app gets the ''VALIDATION'' status from the OCR service
		+	# Now there are two possible events:
		+	## The OCR service sets the status to ''CANCELLED'' when the document has been cancelled by the manual validator
		+	## The OCR service sets the status to ''DONE'' when the document was successfully validated by the manual validator
		+	#: The OCR service also pokes the given callback URL saying there was a statusupdate
		+	# Your app gets the ''CANCELLED'' or ''DONE'' status from the OCR service
		+	# Your app gets the UBL data of the document from the OCR service
−	GetUserInfo	+	==BilltoBox usecase==
		+	<br/>{{note|This is only applicable for companies for which the Value Added Service OCR & Scanning is set to Onea in BilltoBox.}}<br/>
−	Fetch the user information, such as:	+	===Manual validation===
		+	<br/>{{warning|This chapter is only applicable for manual validators.}}<br/>
		+
		+	Please see the link to [https://drive.google.com/open?id=1uAiJ43-Gk2v3waDz2VeUxGUsrzcOYSok Google Drive] for dictionaries, example invoices, etc.
−	- Number of OCR actions per OCR type	+	====Login====
		+	The OCR service validation is located at [https://ocr.onea.be https://ocr.onea.be]. Login with your provided credentials. Once logged in, you will be redirected to the archive screen where you are able to see a list of all documents that still need to be validated.
−	- Number of available credits per OCR type	+	<br/>[[image:ocr_archive.png|link=|900px]]<br/>
		+	<br/>
		+	====Validation====
		+	Click the [[image:view_ok.png|link=]] [[image:view_notok.png|link=]] "View the document" button to open a document.
−	- Allowed OCR types	+	<br/>[[image:ocr_document_view.png|link=|900px]]<br/>
		+	<br/>
−	Parameters: None.	+	=====Available fields=====
		+	<br/>
		+	Available header fields:
		+	* '''Company VAT''': the VAT number for the own company. '''This should always be prefilled by BilltoBox!'''
		+	* '''Invoice type''': the type of the invoice. Possible values are:
		+	** ''F'': invoice
		+	** ''C'': credit note
		+	* '''Supplier/customer''': the supplier from who the invoice came / the customer for which the invoice is intended.
		+	* '''Supplier/customer VAT''': the VAT ID of the supplier/customer. '''This field is read-only and cannot be edited.'''
		+	* '''Supplier/customer IBAN''': the IBAN of the supplier/customer. '''This field is read-only and cannot be edited.'''
		+	* '''Supplier/customer TRN''': the TRN of the supplier/customer. '''This field is read-only and cannot be edited.'''
		+	* '''Invoice number''': the number of the invoice.
		+	* '''Invoice date''': the date of the invoice.
		+	* '''Due date''': the date before which you have to pay the invoice.
		+	* '''OGM''': the structured bank transfer code.
		+	* '''Currency''': the currency used for the invoice.
		+	* '''Net amount''': the net amount of the invoice.
		+	* '''VAT amount''': the VAT amount.
		+	* '''Total amount''': the total amount of the invoice.
		+	<br/>
		+	Available posting line fields:
		+	* '''Net amount''': the amount of the booking line. The default value is the net amount entered on the header.
		+	* '''VAT %''': the VAT percentage used on the document.
		+	* '''Line VAT amount''': the VAT amount for this line.
−	Return: XML schema with the user information.	+	<br/>{{info|Fields shown in red are mandatory and need to be filled in to be able to validate a document. A field is shown in yellow when the recognition server is not sure of the value it found. These fields always need to be validated.}}<br/>
−	'''PostDocument'''	+	=====Available buttons=====
		+	<br/>
		+	* [[image:back.png|link=]] '''"Close"''': close the current opened document.
		+	<br/>{{warning|If there are unsaved changes, these will be lost!}}<br/>
		+	* [[image:save.png|link=]] '''"Save"''': save the changes you made in the document.
		+	* [[image:remove.png|link=]] '''"Cancel document"''': cancel the current document.
		+	* [[image:yes.png|link=]] '''"Validate"''': validate the current document.
		+	* [[image:deliveryrule.png|link=]] '''"Send with email"''': send the current opened document as an e-mail attachment to a certain e-mail address. '''This is only available for incoming invoices!'''
		+	<br/>
		+	When cancelling a document, you need to provide a reason why you think this document should be cancelled. There are multiple possibilities to choose from:
		+	* '''No invoice found''': select this one if the document isn't an invoice.
		+	* '''Other''': select this one if you think for some reason the document should be cancelled. Please provide a comment explaining why you think the document should be cancelled.
		+	<br/>[[image:cancel_document.png|link=]]<br/>
		+	<br/>
−	Post a new document to the OCR service, which should be sent to the OCR engine.	+	=====Supplier/customer validation=====
		+	When no existing supplier/customer could be found with the data found on the invoice, the system gives you the possibility to add a new one.
−	Parameters: Base64 encoded document or upload id.	+	<br/>{{info|The supplier/customer field can be shown in orange, this means that there is already a proposal for creating the supplier/customer.}}<br/>
−	Return: XML schema with the unique key of the document in the OCR service.	+	When a supplier is already selected (either by the recognition server, or by manually selecting one), you still can edit its data.<br/>
		+	Use the [[image:add.png|link=]] "Add" or the [[image:edit.png|link=]] "Edit" button to add or edit the supplier/customer.
		+	<br/>
		+	<br/>{{warning|Don't use the "X" icon in the supplier/customer field in Internet Explorer/Edge for clearing the field! This will not be picked up by the system and the already entered value will be sent to BilltoBox!}}
		+	<br/>
		+	======Available fields======
		+	* '''Active''': should the supplier/customer be active or not?
		+	* '''Code''': the code for the supplier/customer. '''This field cannot be changed!'''
		+	* '''Name''': the name for the supplier/customer.
		+	* '''External ID''': the ID for the supplier/customer in an external package. '''This field should not be used!'''
		+	* '''Aliases''': the aliases for the supplier/customer.
		+	* '''VAT ID''': the VAT ID for the supplier/customer.
		+	* '''Bank account 1''': the bank account (IBAN) for the supplier/customer.
		+	* '''E-mail''': the e-mail address for the supplier/customer.
		+	* '''Website''': the website for the supplier/customer
		+	<br/>[[image:ocr_partner_edit.png|link=|400px]]<br/>
		+	<br/>
−	'''GetDocumentStatus'''	+	====Reports====
		+	To view the available reports for the OCR environment, follow these steps:
−	Check the status of a certain document in the OCR service.	+	# Open the reports via this [https://jasperreports.onea.be/jasperserver-pro url]
		+	# Login with your credentials
		+	# Open the list of reports via ''View - Repository'' in the menu on top
		+	# Select the ''ocr'' folder located in ''Root - Organizations''
		+	# Select the report you would like to see on the right
−	Parameters: Unique key of a document in the OCR service.	+	Following reports are availalbe:
		+	* '''OCR field recognition''': view the recognition rate for the different fields per document
		+	* '''OCR field recognition (chart)''': view the recognition rate for the different fields in a chart
		+	* '''OCR percentages''': view the overall recognition rate percentages and how long it takes to process documents
		+	* '''OCR volumes (chart)''': view a chart of the documents which entered the system and were processed on a specific day
		+	<br/>
−	Return: XML schema with the status of the document in the OCR service.	+	===VAT lines===
		+	====1Archive used as pre-accounting====
		+	When 1Archive is configured as a pre-accounting platform, the master data resides in the corresponding accountancy software. This means that the VAT values used for booking are defined as standard values on either supplier and/or GL account in the accounting software.
		+	When receiving multiple VAT lines and/or multiple VAT regimes from the community platform, an override will be made by 1Archive using the default values used on supplier and/or account level.
−	'''GetDocumentResult'''	+	====Use case scenario when using STP (Straight Through Processing)====
		+	Within 1Archive you can choose to define suppliers on STP = true to enable direct booking. Whenever multiple VAT lines are applicable, only one line will be transferred to the accounting software. If needed, a correction has to be made in the accountancy software.
		+	<br/>{{info|Use STP for suppliers where only onea VAT regime is applicable. When multiple VAT regimes are needed, a correction of the booking can be made in 1Archive or in the accountancy software.}}<br/>
		+	1Archive is using the API of the accountancy software to integrate, which is different from uploading invoices using the accountancy software XML import modules.
−	Get the resulting data of the OCR process, based on the document id in the OCR service.	+	===FAQ===
		+	====Net and VAT amount fields are shown red====
		+	No tolerance is taken into account when amounts are entered into the document. ''"Net amount"'' + ''"VAT amount"'' should always equal ''"Total amount"''. When the fields are shown red, this means that the sum is not correct and the amounts need to be updated.
		+	<br/>
		+	In case the fields are shown red and they are the same amounts as on the image, you can change the ''"Net amount"'' field so the sum equalizes.
		+	<br/> {{warning|<u>'''Never ever'''</u> change the ''"VAT amount"'' and ''"Total amount"'' fields to make sure the sum equalizes!}}<br/>
		+	<br/> [[image:ocr_amount_validation.png|link=|900px]]<br/>
		+	<br/>
−	Parameters: Unique key of a document in the OCR service.	+	====French invoices====
		+	When a French invoice enters the system and the supplier on this invoice has a Dutch name, you are allowed to edit the name and add the French variant to it.
−	Return: File with the OCR result (PDF with embedded data, XML, TXT, …).	+	'''For example''':<br/>
		+	# An invoice for "Agence alimentaire fédérale" enters the system, but the supplier/customer is not recognized
		+	# You check the VAT number on the invoice and find a supplier/customer called "Federaal voedselagentschap"
		+	# You check the website of "Federaal voedselagentschap" to see if there is a French variant of the company
		+	# You see there is a French variant
		+	# You are now allowed to edit the name of the supplier/customer to "Federaal voedselagentschap/Agence alimentaire fédérale"
−	<code>	+	<br/>{{warning|If there is no French variant of the company, you are not allowed to edit the name! In that case, you should create a new supplier/customer.}}<br/>
−	<OCRRESULT xmlns:abbyy="http://www.abbyy.com/FineReader_xml/FineReader10-schema-v1.xml">
−	<TEXTS>
−	<TEXT l="2140" t="15" r="2246" b="40">000529</TEXT>
−	<TEXT l="273" t="294" r="635" b="337">ES FINANCE</TEXT>
−	<TEXT l="273" t="349" r="723" b="377">BNP PARIBAS GROUP</TEXT>
−	<TEXT l="1405" t="524" r="1805" b="541">1225/1307-1/1-8162660001 -1277-001458</TEXT>
−	<TEXT l="1476" t="679" r="1664" b="710">ONEA NV</TEXT>
−	<TEXT l="1476" t="725" r="2198" b="756">OTTERGEMSESTEENWEG-ZUID 731</TEXT>
−	<TEXT l="1475" t="772" r="1695" b="804">9000 </TEXT>
−	…
−	<TEXT l="1407" t="2187" r="1559" b="2216">Papier van</TEXT>
−	<TEXT l="1313" t="2220" r="1653" b="2243">verantwoorde herkomst</TEXT>
−	<TEXT l="1343" t="2274" r="1624" b="2305">FSC® C011145</TEXT>
−	</TEXTS>
−	<BARCODES/>
−	</OCRRESULT>
−	</code>

---

Latest revision as of 07:53, 14 September 2021

1 General

The OCR service is used for recognizing and validating documents coming from an external source. Your app can make use of it by sending PDF documents to Onea Transfer (documentation can be found here).
Those documents will automatically be sent through our recognition server, where they will await validation by a manual validator.

Schematic overview of the flow:

1. Your app is sending a PDF document to the OCR service
2. The OCR service sets the status to PENDING and sends a uuid back
3. The OCR service sets the status to PROCESSING when the document is successfully received and ready to process

   The OCR service also pokes the given callback URL saying there was a statusupdate

4. Your app gets the PROCESSING status from the OCR service
5. The OCR service sets the status to VALIDATION when the document is successfully processed and ready for manual validation

   The OCR service also pokes the given callback URL saying there was a statusupdate

6. Your app gets the VALIDATION status from the OCR service
7. Now there are two possible events:
   1. The OCR service sets the status to CANCELLED when the document has been cancelled by the manual validator
   2. The OCR service sets the status to DONE when the document was successfully validated by the manual validator

   The OCR service also pokes the given callback URL saying there was a statusupdate

8. Your app gets the CANCELLED or DONE status from the OCR service
9. Your app gets the UBL data of the document from the OCR service

2 BilltoBox usecase

2.1 Manual validation

Please see the link to Google Drive for dictionaries, example invoices, etc.

2.1.1 Login

The OCR service validation is located at https://ocr.onea.be. Login with your provided credentials. Once logged in, you will be redirected to the archive screen where you are able to see a list of all documents that still need to be validated.

2.1.2 Validation

Click the "View the document" button to open a document.

2.1.2.1 Available fields

Available header fields:

• Company VAT: the VAT number for the own company. This should always be prefilled by BilltoBox!
• Invoice type: the type of the invoice. Possible values are:
  • F: invoice
  • C: credit note
• Supplier/customer: the supplier from who the invoice came / the customer for which the invoice is intended.
• Supplier/customer VAT: the VAT ID of the supplier/customer. This field is read-only and cannot be edited.
• Supplier/customer IBAN: the IBAN of the supplier/customer. This field is read-only and cannot be edited.
• Supplier/customer TRN: the TRN of the supplier/customer. This field is read-only and cannot be edited.
• Invoice number: the number of the invoice.
• Invoice date: the date of the invoice.
• Due date: the date before which you have to pay the invoice.
• OGM: the structured bank transfer code.
• Currency: the currency used for the invoice.
• Net amount: the net amount of the invoice.
• VAT amount: the VAT amount.
• Total amount: the total amount of the invoice.

Available posting line fields:

• Net amount: the amount of the booking line. The default value is the net amount entered on the header.
• VAT %: the VAT percentage used on the document.
• Line VAT amount: the VAT amount for this line.

2.1.2.2 Available buttons

• "Close": close the current opened document.

• "Save": save the changes you made in the document.
• "Cancel document": cancel the current document.
• "Validate": validate the current document.
• "Send with email": send the current opened document as an e-mail attachment to a certain e-mail address. This is only available for incoming invoices!

When cancelling a document, you need to provide a reason why you think this document should be cancelled. There are multiple possibilities to choose from:

• No invoice found: select this one if the document isn't an invoice.
• Other: select this one if you think for some reason the document should be cancelled. Please provide a comment explaining why you think the document should be cancelled.

2.1.2.3 Supplier/customer validation

When no existing supplier/customer could be found with the data found on the invoice, the system gives you the possibility to add a new one.

When a supplier is already selected (either by the recognition server, or by manually selecting one), you still can edit its data.
Use the "Add" or the "Edit" button to add or edit the supplier/customer.

2.1.2.3.1 Available fields

• Active: should the supplier/customer be active or not?
• Code: the code for the supplier/customer. This field cannot be changed!
• Name: the name for the supplier/customer.
• External ID: the ID for the supplier/customer in an external package. This field should not be used!
• Aliases: the aliases for the supplier/customer.
• VAT ID: the VAT ID for the supplier/customer.
• Bank account 1: the bank account (IBAN) for the supplier/customer.
• E-mail: the e-mail address for the supplier/customer.
• Website: the website for the supplier/customer

2.1.3 Reports

To view the available reports for the OCR environment, follow these steps:

1. Open the reports via this url
2. Login with your credentials
3. Open the list of reports via View - Repository in the menu on top
4. Select the ocr folder located in Root - Organizations
5. Select the report you would like to see on the right

Following reports are availalbe:

• OCR field recognition: view the recognition rate for the different fields per document
• OCR field recognition (chart): view the recognition rate for the different fields in a chart
• OCR percentages: view the overall recognition rate percentages and how long it takes to process documents
• OCR volumes (chart): view a chart of the documents which entered the system and were processed on a specific day

2.2 VAT lines

2.2.1 1Archive used as pre-accounting

When 1Archive is configured as a pre-accounting platform, the master data resides in the corresponding accountancy software. This means that the VAT values used for booking are defined as standard values on either supplier and/or GL account in the accounting software. When receiving multiple VAT lines and/or multiple VAT regimes from the community platform, an override will be made by 1Archive using the default values used on supplier and/or account level.

2.2.2 Use case scenario when using STP (Straight Through Processing)

Within 1Archive you can choose to define suppliers on STP = true to enable direct booking. Whenever multiple VAT lines are applicable, only one line will be transferred to the accounting software. If needed, a correction has to be made in the accountancy software.

1Archive is using the API of the accountancy software to integrate, which is different from uploading invoices using the accountancy software XML import modules.

2.3 FAQ

2.3.1 Net and VAT amount fields are shown red

No tolerance is taken into account when amounts are entered into the document. "Net amount" + "VAT amount" should always equal "Total amount". When the fields are shown red, this means that the sum is not correct and the amounts need to be updated.
In case the fields are shown red and they are the same amounts as on the image, you can change the "Net amount" field so the sum equalizes.

2.3.2 French invoices

When a French invoice enters the system and the supplier on this invoice has a Dutch name, you are allowed to edit the name and add the French variant to it.

For example:

1. An invoice for "Agence alimentaire fédérale" enters the system, but the supplier/customer is not recognized
2. You check the VAT number on the invoice and find a supplier/customer called "Federaal voedselagentschap"
3. You check the website of "Federaal voedselagentschap" to see if there is a French variant of the company
4. You see there is a French variant
5. You are now allowed to edit the name of the supplier/customer to "Federaal voedselagentschap/Agence alimentaire fédérale"