MySQL 5.7.x unable to export sql output file (outfile)


One of my colleague has asked about one strange issue where he created a shell script to export data from mysql in *.csv format at specific location and encountered issue while writing the files. Below is the summary of the same-

Problem
Cannot output MySQL data to file

root@srv1:/MySQLBackup# ./script1.sh 
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1045 (28000) at line 1: Access denied for user 'user'@'%' (using password: YES)
mv: cannot stat '/created_files/data.csv': No such file or directory
sed: can't read /created_files/data2020-04-15.csv: No such file or directory
End of the Script

The MySQL server is running with the --secure-file-priv option so it cannot execute this statement
mv: cannot stat '/created_files/data.csv': No such file or directory

script1.sh
script1.sh
master_db_user=user1
master_db_passwd=user1Pwd
master_db_port=3306
master_db_host=localhost
master_db_name=user1db
Now=$(date -d "yesterday" '+%Y-%m-%d')
mysql -u$master_db_user -p$master_db_passwd -D$master_db_name -e "select * from rb1_details into outfile '/created_files/data.csv' fields terminated by ',' lines terminated by '\n';"
mv /created_files/data.csv /created_files/collection$Now.csv

Cause
MySQL server has been started with --secure-file-priv option which basically limits from which directories you can load files using LOAD DATA INFILE.
You may use SHOW VARIABLES LIKE "secure_file_priv"; to see the directory that has been configured.

mysql> SHOW VARIABLES LIKE "secure_file_priv";
+------------------+-----------------------+
| Variable_name    | Value                 |
+------------------+-----------------------+
| secure_file_priv | /var/lib/mysql-files/ |
+------------------+-----------------------+

Solution
Following possible solutions can be applied-
  1. Move your file to the directory specified by secure-file-priv.
  2. Disable secure-file-priv. This must be removed from startup and cannot be modified dynamically. To do this check your MySQL start up parameters (depending on platform) and my.ini.
  3. Change OUTPUT directory to the directory set for variable secure_file_priv and use your shell script to move files from here to required location.
I have use 3rd solution and now the shell script will look like-

Revised script1.sh
master_db_user=user1
master_db_passwd=user1Pwd
master_db_port=3306
master_db_host=localhost
master_db_name=user1db
Now=$(date -d "yesterday" '+%Y-%m-%d')
mysql -u$master_db_user -p$master_db_passwd -D$master_db_name -e "select * from rb1_details into outfile '/var/lib/mysql-files/data.csv' fields terminated by ',' lines terminated by '\n';"

mv file '/var/lib/mysql-files/data.csv /created_files/collection$Now.csv

This has solved the problem!


Comments

Popular posts from this blog

Oracle SOA Suite- Implementing Email Notification

Oracle SOA Suite 12c- PKIX path building failed & unable to find valid certification path to requested target

Migration of Oracle SOA Suite Composite from 11g to 12c