Monday, December 21, 2015

Enable External Purge in Sterling Integrator

Starting an External Purge

Procedure

  1. If you have a cluster installation, make sure that you are not running the external purge on another node of the cluster. If you are running the external purge on more than one node, it will cause database locks. For more information, see Monitoring an External Purge.
    Important: It is the customer's responsibility to keep exactly one of these external purge processes running in the cluster. If you need to shut down the hardware that is running the external purge, you must run the external purge on a different node.
  2. Disable the default Purge service. Click Deployment Schedules, search for PurgeService, and then clear the Enabled checkbox for PurgeService.
    Note: If you have customized the Schedule_PurgeService business process, you need to add your customized code to the new Schedule_PurgeService business process. The new Schedule_PurgeService business process includes a warning message about running the external purge while the scheduled new purge is enabled.
  3. Turn off the "Important Schedules are Disabled" email notifications by adding the following line to the customer_overrides.properties file. This removes the Purge Service SQL query for the schedule.
    resourceMonitor.ScheduleMonitor.propertyValue.1=SELECT STATUS, SERVICENAME FROMSCHEDULE WHERE SERVICENAME IN ('BackupService','IndexBusinessProcessService','AssociateBPsToDocs'
    ,'BPRecovery','BPLinkagePurgeService')
  4. Do the following:
    • In Windows, do one of the following:
      • Run the command startExternalPurgeWindowsService.cmd from the INSTALL_DIR\bin directory.
      • Run the Windows service. Click Control Panel > Administrative Tools > Services, right-click IBM Sterling B2B Integrator External Purge at BASE_PORT, and click Start.
    • In UNIX, run the command control_extpurge.sh start from the INSTALL_DIR/bin directory.
    • In iSeries, run the command
      SBMJOB CMD (QSH CMD('cd INSTALL_DIR/bin; ./ExternalPurgeConsole.sh -clearlocks > INSTALL_DIR/logs/extpurge.log')) JOB(EXTPURGE)

Remote FTP Directory Sync using lftp

http://www.cyberciti.biz/faq/lftp-mirror-example/

$lftp user@my.ftp.com
$mirror -c source target

Reverse Sync to Server:
Change local directory to  where you saved the files /home/somedirectory
$lcd /home/somedirectory
$mirror -R

Friday, December 11, 2015

Enterprise Logging with Logstash

Basic Flow of Logstash?

LS Agent(s) (Installed on App Server(s)) --> RabbitMQ --> LS Server --> Elasticsearch Server <-- Kibana <-- end user.

So which scripting language is recommended to parse the logs or data?

There is a bunch of plugins for logstash but mostly we use the grok filter.
It takes grok expressions which can be the special grok or even ruby regexes with named captures.
If you need help building patterns to match your logs, you will find the http://grokdebug.herokuapp.com and http://grokconstructor.appspot.com/ applications quite useful!

We need to include Puppet in setup or we can manage with out puppet?

Puppet manages LS Agents on all servers. We create filters and apply them based off the server hostgroup and also input files.

Videos on Logstash:





Download Logstash:



10 Management Tips