Recently, I was working on a POC, which involved coherence as one of the components in building OEP application.
There was a stream of events coming up from CSV and I had to query coherence cache for correlation. While building the application, I receved the below error
Invalid statement: "alter query >>GroupingQuery<< start"
Description: specified predicate requires full scan of external source which is not supported
Cause: specified predicate requires full scan of external source which is not supported
Action: please modify the join predicate
As per the documentation, the above error comes up if cache key is not used in the query, but in our case, it is one of the parameters within the query.
My CQL query was
<query id="GroupingQuery"><![CDATA[
select c.posid, c.tid as totalCount, c.tvalue as totalAmount, p.posname as POSNAME, p.poscity as POSCITY
from CSVIntoProcessor [now] as c, Pos as p
where p.posid<!-- java.lang.Integer -> = c.posid<!-- int ->]]></query>
Pos is my coherence & Posid was my coherence key.
After spending hrs in fixing this, I was able to fix this which was nothing but a data type issue.
To debug I first enable the cache logs
1. Go to ORACLE_CEP_HOME/user_projects/domains/DOMAIN_DIR/servername/config & edit config.xml & change the default values from Notice to Debug
<entry>
<key>Cache</key>
<value> Debug </value>
</entry>
<entry>
<key>CQLProcessor</key>
<value> Debug </value>
</entry>
<entry>
<key>CQLEngine</key>
<value> Debug </value>
</entry>
<entry>
<key>CQLEngineTrace</key>
<value>Debug</value>
</entry>
2. Restart the server and check the logs after deploying your app and look for the below lines
####<May 23, 2015 8:29:52 PM PDT> <Info> <CQLProcessor> <> <defaultserver> <SpringOsgiExtenderThread-154> <> <> <> <1464060592137> <BEA-000000> <execute DDL [register external relation Pos(posid java.lang.Integer, posname char(2147483647), poscity char(2147483647), _this oracle.coherence.Pos)]>
####<May 23, 2016 8:29:52 PM PDT> <Info> <CQLProcessor> <> <defaultserver> <SpringOsgiExtenderThread-154> <> <> <> <1464060592168> <BEA-000000> <execute DDL [register stream channel(posId int, tDate java.util.Date, tValue float, tId int, cId int, _this com.oracle.oep.Transactions) is system timestamped]>
3. Carefully observe the key data types. The external relation pos key is java.lang.Integer, where as the key in my stream channel is posId which is int. (Shown this in red in the error message)
4. Change the data type of the external schema from java.lang.Integer to int and redeply the app, the issue should be fixed. Ideally , that shouldn't have been the problem since java offers auto-boxing but never the less, I am glad that this worked.
Happy Learning!!
Tuesday, May 24, 2016
OEP - Hello World
This blog will walk you thru building a very basic OEP application. components. So this application reads a csv file. The csv file contains list of transactions and contains data like TransactionID, CustomerID, TrasactionValue. In this blog, we will be reading the CSV file and output it to the console.
1. Create new OEP Jdeveloper Project
2. Create supporting java classes
i. Right click on the OEP Project and select "Create New Java Class"
ii. Create a Class named "Transaction" & create private members within it (trasactionID:int, customerId:int, transactionValue:float). Create the setters/getters with it.
package demoprj;
public class Transaction {
int transactionId;
float transactionValue;
int customerId;
public Transaction() {
super();
}
public void setTransactionId(int transactionId) {
this.transactionId = transactionId;
}
public int getTransactionId() {
return transactionId;
}
public void setTransactionValue(float transactionValue) {
this.transactionValue = transactionValue;
}
public float getTransactionValue() {
return transactionValue;
}
public void setCustomerId(int customerId) {
this.customerId = customerId;
}
@Override
public String toString() {
// TODO Implement this method
return new Integer(transactionId).toString();
}
public int getCustomerId() {
return customerId;
}
}
2. Create supporting Event Types
i. Click on the EPN Diagram from the left navigation and click on Event Type Tab & Click on the + icon to create new event Type
ii. Create a new event type named "Transaction" and associated the recently created Transaction java class. Save the project.
3. Drag CSVInbound from the advanced adapter to the EPN Diagram
i. This opens up the CSVInbound configuration screen. Leave the default as its .i.e. Adapter ID: csv-inbound-adapter and configuration location as: processes.xml & Click Next
ii. Select the event type as "Transaction", Source URL as "file:///demo/transaction.csv (this would change depending upon your environment), Event Interval as 1 & Initial Delay as 2 & Click Finish
6. The final step is to add a Processor. The process would be added between the channel & output channel. The Oracle CQL code queries the events sent to the processor from Channel. In a real-world scenario, this component typically executes additional and possibly much more processing of the events from the stream, such as selecting a subset of events based on a property value, grouping events, and so on using Oracle CQL.
i. Drag Processor component on the EPN Diagram.
ii. Without changing the default values click Ok (Processor ID: processor & FileName: Processor.xml)
iii. Right click on the dropped component and click on "Go to Configuration Source"
iv. Make sure channel name is correct. The correct query should be
<rules>
<query id="ExampleQuery"><![CDATA[
select * from channel [now] ]]></query>
</rules>
7. Connect CSV Inbound adapter to channel,channel to processor, processor to outputchannel and output channel to bean., & refresh it. The EPN diagram should look like the below
8. Right-click the OEP project and select Deploy > New Deployment Profile.
Monitor the Server Console, you should be able to see the output as shown below
Sample Content of CSV file being used in this blog
transactionId,transactionValue,customerId
1,10.5,5001
2,10.5,5001
3,10.5,5001
4,10.5,5002
5,10.5,5004
1. Create new OEP Jdeveloper Project
2. Create supporting java classes
i. Right click on the OEP Project and select "Create New Java Class"
ii. Create a Class named "Transaction" & create private members within it (trasactionID:int, customerId:int, transactionValue:float). Create the setters/getters with it.
package demoprj;
public class Transaction {
int transactionId;
float transactionValue;
int customerId;
public Transaction() {
super();
}
public void setTransactionId(int transactionId) {
this.transactionId = transactionId;
}
public int getTransactionId() {
return transactionId;
}
public void setTransactionValue(float transactionValue) {
this.transactionValue = transactionValue;
}
public float getTransactionValue() {
return transactionValue;
}
public void setCustomerId(int customerId) {
this.customerId = customerId;
}
@Override
public String toString() {
// TODO Implement this method
return new Integer(transactionId).toString();
}
public int getCustomerId() {
return customerId;
}
}
iii. Create another class named "TransactionListener". This class implements com.bea.wlevs.ede.api.StreamSink.
package demoprj;
import com.bea.wlevs.ede.api.EventRejectedException;
import com.bea.wlevs.ede.api.StreamSink;
public class TransactionListener implements StreamSink {
public TransactionListner() {
super();
}
@Override
public void onInsertEvent(Object object) throws EventRejectedException {
if (object instanceof Transaction) {
Transaction transactionEvent = (Transaction) object;
System.out.println("Event: " + transactionEvent);
}
}
}
2. Create supporting Event Types
i. Click on the EPN Diagram from the left navigation and click on Event Type Tab & Click on the + icon to create new event Type
ii. Create a new event type named "Transaction" and associated the recently created Transaction java class. Save the project.
3. Drag CSVInbound from the advanced adapter to the EPN Diagram
i. This opens up the CSVInbound configuration screen. Leave the default as its .i.e. Adapter ID: csv-inbound-adapter and configuration location as: processes.xml & Click Next
ii. Select the event type as "Transaction", Source URL as "file:///demo/transaction.csv (this would change depending upon your environment), Event Interval as 1 & Initial Delay as 2 & Click Finish
iii. You should see the csv-inbound-adapter in your EPN diagram as shown in the diagram.
4. Configure: Channels is away by which you can transfer events from one stage to another in the EPN.
i. Drag Channel from EPN components to EPN Page
ii. Select eventType as Transaction and leave default values to channelID
iii. Re Do the above to steps and name the channel ID as output channel & event type to Transaction.
5. Configure Beans: Beans are listener events, that receive events from the channel and works on those events. In my blog ,I am going to print the information to the console.
i. Drag Bean to EPN diagram
ii. Select bean class to "demoprj.TransactionListner" & leave the bean ID to default
6. The final step is to add a Processor. The process would be added between the channel & output channel. The Oracle CQL code queries the events sent to the processor from Channel. In a real-world scenario, this component typically executes additional and possibly much more processing of the events from the stream, such as selecting a subset of events based on a property value, grouping events, and so on using Oracle CQL.
i. Drag Processor component on the EPN Diagram.
ii. Without changing the default values click Ok (Processor ID: processor & FileName: Processor.xml)
iii. Right click on the dropped component and click on "Go to Configuration Source"
iv. Make sure channel name is correct. The correct query should be
<rules>
<query id="ExampleQuery"><![CDATA[
select * from channel [now] ]]></query>
</rules>
7. Connect CSV Inbound adapter to channel,channel to processor, processor to outputchannel and output channel to bean., & refresh it. The EPN diagram should look like the below
8. Right-click the OEP project and select Deploy > New Deployment Profile.
Monitor the Server Console, you should be able to see the output as shown below
transactionId,transactionValue,customerId
1,10.5,5001
2,10.5,5001
3,10.5,5001
4,10.5,5002
5,10.5,5004
OEP - Creating New OEP Project in Jdeveloper
In this blog, you will learn how to create new OEP application using J Developer
1. Open Jdeveloper
2. Click File->New->Application and Select OEP application & Click Ok
3. Give the application details like application name, prefix, directory structure etc. & Click Next
4. Give Project details like project name & directory. Note the project features that has been added to the application. You should see Java, XML & OEP Suite in the project feature section
7. Click Ok->Finish. This will create new OEP Project
1. Open Jdeveloper
2. Click File->New->Application and Select OEP application & Click Ok
3. Give the application details like application name, prefix, directory structure etc. & Click Next
4. Give Project details like project name & directory. Note the project features that has been added to the application. You should see Java, XML & OEP Suite in the project feature section
5. Configure Java Settings. The default packaged mentioned here will used to store all OEP related java classes.
6. In the next window, select Empty OEP Project & Click on + icon to create OEP Server Connection. Give your OEP server details which could be either remote or local.
Subscribe to:
Posts (Atom)