2 Q&As

A survival guide covering the most common (and painful) Frappe issues encountered in the wild, from packet sequence errors to database access denied nightmares.

Note

Common problems and their workarounds

🐛 Problem 1: Packet Sequence Error

First check for packet sequence problem by running below command in console.

# open python console
bench --site dev.localhost console --autoreload

# import samcore
import samcore.samcore_api as sca

# run this 2 times — it will give you error; if it does, it is packet sequence problem
sca.ucr_fetch('Administrator','report_4')

Now paste below code in your frappe/database/database.py.

def sql(
		self,
		query: Query,
		values: QueryValues = EmptyQueryValues,
		*,
		as_dict=0,
		as_list=0,
		debug=0,
		ignore_ddl=0,
		auto_commit=0,
		update=None,
		explain=False,
		run=True,
		pluck=False,
		as_iterator=False,
	):
		"""Execute a SQL query and fetch all rows.

		:param query: SQL query.
		:param values: Tuple / List / Dict of values to be escaped and substituted in the query.
		:param as_dict: Return as a dictionary.
		:param as_list: Always return as a list.
		:param debug: Print query and `EXPLAIN` in debug log.
		:param ignore_ddl: Catch exception if table, column missing.
		:param auto_commit: Commit after executing the query.
		:param update: Update this dict to all rows (if returned `as_dict`).
		:param run: Return query without executing it if False.
		:param pluck: Get the plucked field only.
		:param explain: Print `EXPLAIN` in error log.
		:param as_iterator: Returns iterator over results instead of fetching all results at once.
				This should be used with unbuffered cursor as default cursors used by pymysql and postgres
				buffer the results internally. See `Database.unbuffered_cursor`.
		Examples:

				# return customer names as dicts
				frappe.db.sql("select name from tabCustomer", as_dict=True)

				# return names beginning with a
				frappe.db.sql("select name from tabCustomer where name like %s", "a%")

				# values as dict
				frappe.db.sql("select name from tabCustomer where name like %(name)s and owner=%(owner)s",
						{"name": "a%", "owner":"test@example.com"})

		"""
		if isinstance(query, MySQLQueryBuilder | PostgreSQLQueryBuilder):
			frappe.log("Use run method to execute SQL queries generated by Query Builder")

		debug = debug or getattr(self, "debug", False)
		query = str(query)
		if not run:
			return query

		# remove whitespace / indentation from start and end of query
		query = query.strip()

		# replaces ifnull in query with coalesce
		query = IFNULL_PATTERN.sub("coalesce(", query)

		if not self._conn:
			self.connect()

		# in transaction validations
		self.check_transaction_status(query)
		self.clear_db_table_cache(query)

		if auto_commit:
			self.commit()

		if debug:
			time_start = time()

		if values == EmptyQueryValues:
			values = None
		elif not isinstance(values, tuple | dict | list):
			values = (values,)

		query, values = self._transform_query(query, values)

		if trace_id := get_trace_id():
			query += f" /* FRAPPE_TRACE_ID: {trace_id} */"

		try:
			self._cursor.execute(query, values)
		except Exception as e:
			if self.is_syntax_error(e):
				frappe.log(f"Syntax error in query:\n{query} {values or ''}")

			elif self.is_deadlocked(e):
				raise frappe.QueryDeadlockError(e) from e

			elif self.is_timedout(e):
				raise frappe.QueryTimeoutError(e) from e

			elif self.is_read_only_mode_error(e):
				frappe.throw(
					_(
						"Site is running in read only mode for maintenance or site update, this action can not be performed right now. Please try again later."
					),
					title=_("In Read Only Mode"),
					exc=frappe.InReadOnlyMode,
				)

			# TODO: added temporarily
			elif self.db_type == "postgres":
				traceback.print_stack()
				frappe.log(f"Error in query:\n{e}")
				raise

			elif isinstance(e, self.ProgrammingError):
				if frappe.conf.developer_mode:
					traceback.print_stack()
					frappe.log(f"Error in query:\n{query, values}")
				raise

			if not (
				ignore_ddl
				and (self.is_missing_column(e) or self.is_table_missing(e) or self.cant_drop_field_or_key(e))
			):
				raise

		self.log_query(query, values, debug, explain)
		if debug:
			time_end = time()
			frappe.log(f"Execution time: {time_end - time_start:.2f} sec")

		# if auto_commit:
		# 	self.commit()

		# if not self._cursor.description:
		# 	return ()

		# if as_iterator:
		# 	return self._return_as_iterator(pluck=pluck, as_dict=as_dict, as_list=as_list, update=update)

		# last_result = self._transform_result(self._cursor.fetchall())

		# packet sequence problem solution
		if auto_commit:
			self.commit()
			temp_description = None

		if not self._cursor.description:
			return ()
		else:
			temp_description = self._cursor.description
		
		is_sp = query.strip().lower().startswith("call")
		last_result = self._transform_result(self._cursor.fetchall())
		if is_sp:
			while self._cursor.nextset():
				pass

		if temp_description:
			self._cursor.description = temp_description 
		if as_iterator:
			return self._return_as_iterator(pluck=pluck, as_dict=as_dict, as_list=as_list, update=update)
		# packet sequence problem solution
		
		if pluck:
			last_result = [r[0] for r in last_result]
			self._clean_up()
			return last_result

		# scrub output if required
		if as_dict:
			last_result = self.fetch_as_dict(last_result)
			if update:
				for r in last_result:
					r.update(update)

		elif as_list:
			last_result = self.convert_to_lists(last_result)

		self._clean_up()
		return last_result

Now restart bench. It should solve packet sequence problem.

🐛 Problem 2: Report Data

This is probably because when we setup locally there is no env variables so instead of adding them we have find one other way for it.

Add below connection string in common_site_config.json.

"Sam123": "DRIVER={ODBC Driver 18 for SQL Server};SERVER={10.10.0.123};DATABASE={other};UID={RO};PWD={Read@Only};Encrypt=yes;TrustServerCertificate=yes"

Now you can use this by below string.

conn_str = frappe.conf.get("Sam123")

Add key in samcore/js/cutom/usr_cust_format.js.

Z7GG-XF6E1P-504Q0T-2X6173-156X06-1N1R1M-261V2J-580O73-3B

🐛 Problem 3: Access Denied

pymysql.err.OperationalError: (1045, "Access denied for user '_3b41b400ef4d007a'@'172.19.0.5' (using password: YES)")

> SELECT user, host FROM mysql.user WHERE user = '_3b41b400ef4d007a';
+-------------------+------------+
| User              | Host       |
+-------------------+------------+
| _3b41b400ef4d007a | 10.10.10.6 |
| _3b41b400ef4d007a | localhost  |
+-------------------+------------+
2 rows in set (0.022 sec)

CREATE USER '_3b41b400ef4d007a'@'%' IDENTIFIED BY 'your_password';
GRANT ALL PRIVILEGES ON *.* TO '_3b41b400ef4d007a'@'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;

🐛 Problem 4: Corrupted InnoDB (Table Space Missing)

Run this in your docker container or where you have installed MariaDB.

MYSQL_DATA_DIR="/var/lib/mysql/_3b41b400ef4d007a/"  
cd $MYSQL_DATA_DIR

find . -name "*.frm" | while read frm_file; do
    ibd_file="${frm_file%.frm}.ibd"
    if [ ! -f "$ibd_file" ]; then
        echo "Missing: $ibd_file"
    fi
done

It will give you all corrupted database files.

🐛 Problem 5: Internal Server Error (Might be Case of Database)

While creating site pass --mariadb-user-host-login-scope='%'

bench new-site --mariadb-user-host-login-scope='%' --admin-password=admin --db-name=stv --db-root-username=root --db-root-password=Server_Mariadb_stv@1987 --install-app erpnext --set-default frontend;